[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:16 dist-test-slave-8cs2 krb5kdc[7855](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:16 dist-test-slave-8cs2 krb5kdc[7855](info): set up 2 sockets
Sep 29 02:18:16 dist-test-slave-8cs2 krb5kdc[7855](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260929 02:18:17.517292 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.947s user 0.003s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:17 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:17.676482 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40477
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:40477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:18.006729 7868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:18.007052 7868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:18.007170 7868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:18.014621 7868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:18.014752 7868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:18.014801 7868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:18.014967 7868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:18.015035 7868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:18.025730 7868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:40477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:18.027104 7868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:18.028409 7868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:18.040714 7873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:18.040743 7874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:18.040776 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:18.041493 7868 server_base.cc:1034] running on GCE node
I20260929 02:18:18.042215 7868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:18.043606 7868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:18.044814 7868 hybrid_clock.cc:648] HybridClock initialized: now 1790648298044753 us; error 51 us; skew 500 ppm
I20260929 02:18:18.045190 7868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:18 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648298, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:18.048645 7868 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:18.050172 7868 webserver.cc:460] Webserver started at http://127.7.170.62:35841/ using document root <none> and password file <none>
I20260929 02:18:18.050885 7868 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:18.050992 7868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:18.051278 7868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:18.054311 7868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "90903efd84cd458a8e2aa238af04dcc3"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "4a23a03457edb944e4889ef9912d6424"
server_key_iv: "7d7deffce0a99bf25d826433cafd5ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.055025 7868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "90903efd84cd458a8e2aa238af04dcc3"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "4a23a03457edb944e4889ef9912d6424"
server_key_iv: "7d7deffce0a99bf25d826433cafd5ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.060084 7868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260929 02:18:18.064085 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:18.065619 7868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:18:18.065775 7868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "90903efd84cd458a8e2aa238af04dcc3"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "4a23a03457edb944e4889ef9912d6424"
server_key_iv: "7d7deffce0a99bf25d826433cafd5ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.065972 7868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:18.084698 7868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:18.088828 7868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:18.089099 7868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:18.107529 7868 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:40477
I20260929 02:18:18.107597 7934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:40477 every 8 connection(s)
I20260929 02:18:18.108860 7868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:18.111728 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 7868
I20260929 02:18:18.111990 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:18.112452 7935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:18.113801 7848 external_mini_cluster.cc:1428] Setting key 60098a1e7dc7936ecea2b4d3bb074e0e
I20260929 02:18:18.123165 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3: Bootstrap starting.
I20260929 02:18:18.126422 7935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:18.127404 7935 log.cc:824] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:18.130129 7935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3: No bootstrap required, opened a new log
I20260929 02:18:18.137533 7935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } }
I20260929 02:18:18.137914 7935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:18.138003 7935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90903efd84cd458a8e2aa238af04dcc3, State: Initialized, Role: FOLLOWER
I20260929 02:18:18.138511 7935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } }
I20260929 02:18:18.138718 7935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:18.138829 7935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:18.138991 7935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 0 FOLLOWER]: Advancing to term 1
Sep 29 02:18:18 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:18.141167 7935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } }
I20260929 02:18:18.141597 7935 leader_election.cc:302] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90903efd84cd458a8e2aa238af04dcc3; no voters:
I20260929 02:18:18.142228 7935 leader_election.cc:288] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:18.142339 7940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:18.143858 7940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [term 1 LEADER]: Becoming Leader. State: Replica: 90903efd84cd458a8e2aa238af04dcc3, State: Running, Role: LEADER
I20260929 02:18:18.144395 7940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } }
I20260929 02:18:18.144610 7935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:18.150197 7937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:18.117421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35928 (local address 127.7.170.62:40477)
0929 02:18:18.118215 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:18.118225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:18.119746 (+ 1521us) server_negotiation.cc:407] Connection header received
0929 02:18:18.128774 (+ 9028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:18.128794 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:18.129201 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:18.129541 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:18.132262 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.133729 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.135332 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.136021 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.136203 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:18.141683 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:18.141691 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:18.141704 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:18.141741 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:18.145540 (+ 3799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:18.146463 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:18.146469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:18.146474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:18.146617 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:18.147026 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:18.147031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:18.147035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:18.147570 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:18.147892 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:18.148667 (+ 775us) server_negotiation.cc:299] Negotiation successful
0929 02:18:18.149075 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":383,"threads_started":1}
I20260929 02:18:18.154027 7941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90903efd84cd458a8e2aa238af04dcc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } } }
I20260929 02:18:18.153932 7942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90903efd84cd458a8e2aa238af04dcc3. Latest consensus state: current_term: 1 leader_uuid: "90903efd84cd458a8e2aa238af04dcc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90903efd84cd458a8e2aa238af04dcc3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40477 } } }
I20260929 02:18:18.154312 7941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:18.154351 7942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:18.161900 7947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:18.167301 7947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:18.176752 7947 catalog_manager.cc:1347] Generated new cluster ID: dd1ae08f885e4a059e385e35ed4e34ee
I20260929 02:18:18.176867 7947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:18.188795 7947 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:18.190125 7947 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:18.199476 7947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90903efd84cd458a8e2aa238af04dcc3: Generated new TSK 0
I20260929 02:18:18.200774 7947 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:18.248580 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:18.584884 7961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:18.585197 7961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:18.585291 7961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:18.592638 7961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:18.592749 7961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:18.592880 7961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:18.602578 7961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:18.603782 7961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:18.605230 7961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:18.622767 7966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:18.622838 7969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:18.622855 7967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:18.623690 7961 server_base.cc:1034] running on GCE node
I20260929 02:18:18.624336 7961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:18.625782 7961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:18.627019 7961 hybrid_clock.cc:648] HybridClock initialized: now 1790648298626970 us; error 44 us; skew 500 ppm
I20260929 02:18:18.627345 7961 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:18 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648298, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:18.630514 7961 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:18.631985 7961 webserver.cc:460] Webserver started at http://127.7.170.1:41151/ using document root <none> and password file <none>
I20260929 02:18:18.632673 7961 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:18.632781 7961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:18.633117 7961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:18.636051 7961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "859818a7bd774b70b430ddb53485bf22"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "7285da9816708c0471d57f4615dcd7d9"
server_key_iv: "91c89e00463f01efd3e1aa2a4c039774"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.636747 7961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "859818a7bd774b70b430ddb53485bf22"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "7285da9816708c0471d57f4615dcd7d9"
server_key_iv: "91c89e00463f01efd3e1aa2a4c039774"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.641717 7961 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:18:18.645108 7976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:18.646432 7961 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:18.646575 7961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "859818a7bd774b70b430ddb53485bf22"
format_stamp: "Formatted at 2026-09-29 02:18:18 on dist-test-slave-8cs2"
server_key: "7285da9816708c0471d57f4615dcd7d9"
server_key_iv: "91c89e00463f01efd3e1aa2a4c039774"
server_key_version: "encryptionkey@0"
I20260929 02:18:18.646741 7961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:18.674825 7961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:18.679069 7961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:18.679438 7961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:18.680375 7961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:18.681686 7961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:18.681829 7961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:18.681988 7961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:18.682049 7961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:18.715560 7961 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:40205
I20260929 02:18:18.715643 8088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:40205 every 8 connection(s)
I20260929 02:18:18.716779 7961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:18.724273 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 7961
I20260929 02:18:18.724466 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:18.725061 7848 external_mini_cluster.cc:1428] Setting key 58aff0b23c5aa62e5bff556c3ff6fdf3
Sep 29 02:18:18 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648298, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:18.738718 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:18.719494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:50213 (local address 127.7.170.62:40477)
0929 02:18:18.719945 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:18.719950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:18.721284 (+ 1334us) server_negotiation.cc:407] Connection header received
0929 02:18:18.723145 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:18.723150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:18.723231 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:18.723327 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:18.724855 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.725713 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.727562 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.728085 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.728230 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:18.731794 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:18.731800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:18.731805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:18.731840 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:18.734811 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:18.735908 (+ 1097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:18.735914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:18.735917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:18.736056 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:18.736557 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:18.736562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:18.736565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:18.736896 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:18.737065 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:18.738039 (+ 974us) server_negotiation.cc:299] Negotiation successful
0929 02:18:18.738338 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":220,"threads_started":1}
I20260929 02:18:18.739686 8092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:18.720013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40477 (local address 127.7.170.1:50213)
0929 02:18:18.721130 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:18.721169 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:18.722839 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:18.723564 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:18.723573 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:18.724095 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:18.724699 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:18.724718 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.725894 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.725898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:18.727429 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:18.727445 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:18.728316 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:18.728323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:18.728589 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:18.729241 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:18.729263 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:18.731558 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:18.734965 (+ 3407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:18.735002 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:18.735024 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:18.735757 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:18.736183 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:18.736188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:18.736192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:18.736422 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:18.737142 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:18.737150 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:18.737595 (+ 445us) client_negotiation.cc:769] Sending connection context
0929 02:18:18.737899 (+ 304us) client_negotiation.cc:240] Negotiation successful
0929 02:18:18.738412 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":635,"threads_started":1}
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:18.741684 8089 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40477
I20260929 02:18:18.741969 8089 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:18.742763 8089 heartbeater.cc:511] Master 127.7.170.62:40477 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:18.744993 7900 ts_manager.cc:194] Registered new tserver with Master: 859818a7bd774b70b430ddb53485bf22 (127.7.170.1:40205)
I20260929 02:18:18.746793 7900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:50213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:18.764665 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:19.093993 8095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:19.094266 8095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:19.094390 8095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:19.101475 8095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:19.101642 8095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:19.101814 8095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:19.111436 8095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:19.112612 8095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:19.113926 8095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:19.127022 8103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:19.127108 8100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:19.127082 8101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:19.127658 8095 server_base.cc:1034] running on GCE node
I20260929 02:18:19.128244 8095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:19.129262 8095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:19.130466 8095 hybrid_clock.cc:648] HybridClock initialized: now 1790648299130412 us; error 50 us; skew 500 ppm
I20260929 02:18:19.130755 8095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:19.133590 8095 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:19.134967 8095 webserver.cc:460] Webserver started at http://127.7.170.2:34165/ using document root <none> and password file <none>
I20260929 02:18:19.135571 8095 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:19.135661 8095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:19.135911 8095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:19.138747 8095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "c71f130a3494483f88ffd4464db2cf24"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "20b2858c6f272cc92f38559f6923990c"
server_key_iv: "2b6da30c5697a131ee0131314e3cd152"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.139485 8095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "c71f130a3494483f88ffd4464db2cf24"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "20b2858c6f272cc92f38559f6923990c"
server_key_iv: "2b6da30c5697a131ee0131314e3cd152"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.144574 8095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:18:19.148319 8110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.149963 8095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:19.150104 8095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "c71f130a3494483f88ffd4464db2cf24"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "20b2858c6f272cc92f38559f6923990c"
server_key_iv: "2b6da30c5697a131ee0131314e3cd152"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.150279 8095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:19.162465 8095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:19.166498 8095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:19.166751 8095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:19.167630 8095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:19.169097 8095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:19.169170 8095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.169256 8095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:19.169297 8095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.201634 8095 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:35697
I20260929 02:18:19.201669 8222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:35697 every 8 connection(s)
I20260929 02:18:19.202850 8095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:19.210228 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8095
I20260929 02:18:19.210404 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:19.210784 7848 external_mini_cluster.cc:1428] Setting key 0a98afa6450d06e305127fb54309b326
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:19.223030 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.205398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:46535 (local address 127.7.170.62:40477)
0929 02:18:19.205603 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.205608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.206946 (+ 1338us) server_negotiation.cc:407] Connection header received
0929 02:18:19.208710 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.208714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.208794 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.208877 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.210393 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.211133 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.212664 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.213114 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.213249 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.217649 (+ 4400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.217655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.217658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.217689 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.220044 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.220715 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.220721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.220724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.220886 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.221387 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.221392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.221395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.221687 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.221811 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.222488 (+ 677us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.222712 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:18:19.224006 8225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.205997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40477 (local address 127.7.170.2:46535)
0929 02:18:19.206781 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:19.206833 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:19.208459 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:19.209125 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:19.209134 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:19.209642 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:19.210227 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:19.210243 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.211277 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.211281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:19.212524 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:19.212540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.213329 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.213336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:19.213651 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.214354 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:19.214381 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:19.217436 (+ 3055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:19.220171 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:19.220180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:19.220198 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:19.220611 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:19.221080 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:19.221085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:19.221090 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:19.221278 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:19.221805 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:19.221813 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:19.222153 (+ 340us) client_negotiation.cc:769] Sending connection context
0929 02:18:19.222391 (+ 238us) client_negotiation.cc:240] Negotiation successful
0929 02:18:19.222914 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":348,"threads_started":1}
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:19.225529 8223 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40477
I20260929 02:18:19.225812 8223 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:19.226527 8223 heartbeater.cc:511] Master 127.7.170.62:40477 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
I20260929 02:18:19.227886 7900 ts_manager.cc:194] Registered new tserver with Master: c71f130a3494483f88ffd4464db2cf24 (127.7.170.2:35697)
I20260929 02:18:19.228967 7900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:46535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:19.245918 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:19.572196 8228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:19.572468 8228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:19.572573 8228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:19.579921 8228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:19.580443 8228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:19.580621 8228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:19.590519 8228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40477
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:19.591755 8228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:19.593055 8228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:19.605800 8236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:19.605835 8234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:19.605762 8233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:19.606765 8228 server_base.cc:1034] running on GCE node
I20260929 02:18:19.607394 8228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:19.608256 8228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:19.609447 8228 hybrid_clock.cc:648] HybridClock initialized: now 1790648299609381 us; error 52 us; skew 500 ppm
I20260929 02:18:19.609735 8228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:19.612607 8228 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:19.614061 8228 webserver.cc:460] Webserver started at http://127.7.170.3:44055/ using document root <none> and password file <none>
I20260929 02:18:19.614737 8228 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:19.614830 8228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:19.615082 8228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:19.617995 8228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "9607aaa763f2493fbb022615a4267650"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "9dbe0eb9d1c351c1638f5e2bb95a2008"
server_key_iv: "c6366f5309dcb54f56735fd395c00e1c"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.618666 8228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "9607aaa763f2493fbb022615a4267650"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "9dbe0eb9d1c351c1638f5e2bb95a2008"
server_key_iv: "c6366f5309dcb54f56735fd395c00e1c"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.623972 8228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:18:19.627310 8243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.628607 8228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260929 02:18:19.628731 8228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "9607aaa763f2493fbb022615a4267650"
format_stamp: "Formatted at 2026-09-29 02:18:19 on dist-test-slave-8cs2"
server_key: "9dbe0eb9d1c351c1638f5e2bb95a2008"
server_key_iv: "c6366f5309dcb54f56735fd395c00e1c"
server_key_version: "encryptionkey@0"
I20260929 02:18:19.628893 8228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:19.645888 8228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:19.649966 8228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:19.650219 8228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:19.651185 8228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:19.653010 8228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:19.653086 8228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.653218 8228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:19.653266 8228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:19.684676 8228 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:34853
I20260929 02:18:19.684718 8355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:34853 every 8 connection(s)
I20260929 02:18:19.686064 8228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:19.690922 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8228
I20260929 02:18:19.691104 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:19.691682 7848 external_mini_cluster.cc:1428] Setting key b7942493fbe97beb49a5740193700a22
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:19.706641 8091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.688458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35391 (local address 127.7.170.62:40477)
0929 02:18:19.688669 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.688675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.689877 (+ 1202us) server_negotiation.cc:407] Connection header received
0929 02:18:19.691974 (+ 2097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.691979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.692046 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.692127 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.693990 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.695097 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.697151 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.697549 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.697647 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.700882 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.700887 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.700891 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.700957 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.703378 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.704078 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.704082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.704085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.704250 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.704695 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.704698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.704701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.705071 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.705189 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.706140 (+ 951us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.706322 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:18:19.707733 8358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.689017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40477 (local address 127.7.170.3:35391)
0929 02:18:19.689724 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:19.689765 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:19.691694 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:19.692387 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:19.692398 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:19.693066 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:19.693830 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:19.693855 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.695259 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.695264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:19.697015 (+ 1751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:19.697026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.697726 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.697731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:19.697918 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.698499 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:19.698530 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:19.700651 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:19.703507 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:19.703514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:19.703527 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:19.703945 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:19.704378 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:19.704382 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:19.704384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:19.704579 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:19.705245 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:19.705251 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:19.705633 (+ 382us) client_negotiation.cc:769] Sending connection context
0929 02:18:19.705988 (+ 355us) client_negotiation.cc:240] Negotiation successful
0929 02:18:19.706460 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":292,"threads_started":1}
I20260929 02:18:19.708963 8356 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40477
I20260929 02:18:19.709230 8356 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:19.709913 8356 heartbeater.cc:511] Master 127.7.170.62:40477 requested a full tablet report, sending...
I20260929 02:18:19.711400 7900 ts_manager.cc:194] Registered new tserver with Master: 9607aaa763f2493fbb022615a4267650 (127.7.170.3:34853)
I20260929 02:18:19.712175 7900 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:35391
I20260929 02:18:19.720009 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260929 02:18:19.749768 8089 heartbeater.cc:503] Master 127.7.170.62:40477 was elected leader, sending a full tablet report...
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:19.760581 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.746210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.7.170.1:40205)
0929 02:18:19.747239 (+ 1029us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.747246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.747273 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:18:19.747330 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.747348 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.747557 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.747681 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.749020 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.750107 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.751405 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.751954 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.752047 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.755750 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.755762 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.755776 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.755815 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.758363 (+ 2548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.758911 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.758917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.758920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.759034 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.759318 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.759321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.759323 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.759642 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.759881 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.760063 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.760244 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":262,"threads_started":1}
W20260929 02:18:19.761823 8004 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38996
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:19.809736 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.789246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39006 (local address 127.7.170.1:40205)
0929 02:18:19.789466 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.789472 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.789628 (+ 156us) server_negotiation.cc:407] Connection header received
0929 02:18:19.789777 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.789782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.789904 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.789985 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.791230 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.791935 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.793168 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.793593 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.793695 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.805071 (+ 11376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.805077 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.805080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.805118 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.807465 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.808119 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.808123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.808125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.808245 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.808548 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.808551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.808552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.808792 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.809000 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.809299 (+ 299us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.809459 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:18:19.811194 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 7961
I20260929 02:18:19.823283 8363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.812038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39016 (local address 127.7.170.1:40205)
0929 02:18:19.812180 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.812184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.812532 (+ 348us) server_negotiation.cc:407] Connection header received
0929 02:18:19.812632 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.812636 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.812688 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.812758 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.813852 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.814566 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.815764 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.816170 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.816271 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.818796 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.818800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.818803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.818840 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.821251 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.821855 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.821859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.821861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.821980 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.822227 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.822230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.822231 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.822455 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.822687 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.822894 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.823039 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:18:19.823913 8084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:19.904403 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 7961
I20260929 02:18:19.917932 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8095
Sep 29 02:18:19 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:19.932809 8370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:19.918743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46218 (local address 127.7.170.2:35697)
0929 02:18:19.919238 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:19.919245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:19.919269 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:18:19.919328 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:19.919335 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:19.919536 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:19.919656 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:19.920811 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.921950 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.923185 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:19.923723 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:19.923825 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:19.927650 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:19.927661 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:19.927675 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:19.927707 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:19.930264 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.930858 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.930866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.930868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.930988 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:19.931323 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:19.931327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:19.931329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:19.931643 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:19.931932 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:19.932222 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 02:18:19.932459 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":256,"threads_started":1}
I20260929 02:18:19.934077 8218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:20.016389 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8095
I20260929 02:18:20.033946 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8228
Sep 29 02:18:20 dist-test-slave-8cs2 krb5kdc[7855](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:20.049009 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:20.034924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51824 (local address 127.7.170.3:34853)
0929 02:18:20.035511 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:20.035518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:20.035545 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:18:20.035606 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:20.035612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:20.035813 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:20.035963 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:20.037237 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:20.038254 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:20.039501 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:20.040045 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:20.040154 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:20.043938 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:20.043949 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:20.043963 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:20.043996 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:20.046745 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:20.047374 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:20.047381 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:20.047383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:20.047502 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:20.047769 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:20.047772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:20.047774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:20.048096 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:20.048323 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:20.048501 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 02:18:20.048686 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":292,"threads_started":1}
I20260929 02:18:20.050288 8351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:20.130179 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8228
I20260929 02:18:20.143388 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 7868
I20260929 02:18:20.144488 7930 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:20.242992 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 7868
2026-09-29T02:18:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3720 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:20 dist-test-slave-8cs2 krb5kdc[8377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:20 dist-test-slave-8cs2 krb5kdc[8377](info): set up 2 sockets
Sep 29 02:18:20 dist-test-slave-8cs2 krb5kdc[8377](info): commencing operation
krb5kdc: starting...
W20260929 02:18:21.233152 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.953s user 0.003s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:21 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:21.396613 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44307
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:44307
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:21.703004 8390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:21.703301 8390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:21.703416 8390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:21.710708 8390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:21.710827 8390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:21.710873 8390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:21.710947 8390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:21.711004 8390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:21.720952 8390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:44307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:21.722268 8390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:21.723593 8390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:21.735471 8396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:21.735491 8398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:21.735461 8395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:21.736212 8390 server_base.cc:1034] running on GCE node
I20260929 02:18:21.736997 8390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:21.738407 8390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:21.739616 8390 hybrid_clock.cc:648] HybridClock initialized: now 1790648301739560 us; error 37 us; skew 500 ppm
I20260929 02:18:21.739964 8390 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:21 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648301, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:21.742964 8390 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:21.744288 8390 webserver.cc:460] Webserver started at http://127.7.170.62:36721/ using document root <none> and password file <none>
I20260929 02:18:21.744997 8390 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:21.745108 8390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:21.745394 8390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:21.748245 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "17f9285e02714fc6966f38ee2b127c3e"
format_stamp: "Formatted at 2026-09-29 02:18:21 on dist-test-slave-8cs2"
server_key: "f86cc473ac5943b9e1c49d3fa25168ed"
server_key_iv: "a74557f81f4fe663895ed79ace0c3d31"
server_key_version: "encryptionkey@0"
I20260929 02:18:21.748967 8390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "17f9285e02714fc6966f38ee2b127c3e"
format_stamp: "Formatted at 2026-09-29 02:18:21 on dist-test-slave-8cs2"
server_key: "f86cc473ac5943b9e1c49d3fa25168ed"
server_key_iv: "a74557f81f4fe663895ed79ace0c3d31"
server_key_version: "encryptionkey@0"
I20260929 02:18:21.753790 8390 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:18:21.757119 8405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:21.758370 8390 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:21.758517 8390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "17f9285e02714fc6966f38ee2b127c3e"
format_stamp: "Formatted at 2026-09-29 02:18:21 on dist-test-slave-8cs2"
server_key: "f86cc473ac5943b9e1c49d3fa25168ed"
server_key_iv: "a74557f81f4fe663895ed79ace0c3d31"
server_key_version: "encryptionkey@0"
I20260929 02:18:21.758689 8390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:21.777043 8390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:21.780761 8390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:21.781040 8390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:21.797052 8390 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:44307
I20260929 02:18:21.797089 8456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:44307 every 8 connection(s)
I20260929 02:18:21.798328 8390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:21.800844 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8390
I20260929 02:18:21.801084 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:21.801869 7848 external_mini_cluster.cc:1428] Setting key d246ee5986736993cbeeb715887b42c7
I20260929 02:18:21.802673 8457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:21.814553 8457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e: Bootstrap starting.
Sep 29 02:18:21 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:21.817776 8457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:21.818773 8457 log.cc:824] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:21.821369 8457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e: No bootstrap required, opened a new log
I20260929 02:18:21.822422 8460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:21.803864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.7.170.62:44307)
0929 02:18:21.804585 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:21.804597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:21.804632 (+ 35us) server_negotiation.cc:407] Connection header received
0929 02:18:21.806162 (+ 1530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:21.806185 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:21.806694 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:21.807094 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:21.808499 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:21.809858 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:21.811036 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:21.811720 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:21.811908 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:21.815554 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:21.815561 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:21.815576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:21.815615 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:21.818635 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:21.819293 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:21.819299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:21.819305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:21.819470 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:21.819814 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:21.819817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:21.819819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:21.820330 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:21.820597 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:21.821075 (+ 478us) server_negotiation.cc:299] Negotiation successful
0929 02:18:21.821454 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":318,"threads_started":1}
I20260929 02:18:21.828486 8457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } }
I20260929 02:18:21.828794 8457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:21.828879 8457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17f9285e02714fc6966f38ee2b127c3e, State: Initialized, Role: FOLLOWER
I20260929 02:18:21.829392 8457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } }
I20260929 02:18:21.829582 8457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:21.829669 8457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:21.829782 8457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:21.832118 8457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } }
I20260929 02:18:21.832535 8457 leader_election.cc:302] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 17f9285e02714fc6966f38ee2b127c3e; no voters:
I20260929 02:18:21.833108 8457 leader_election.cc:288] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:21.833223 8462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:21.833869 8462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [term 1 LEADER]: Becoming Leader. State: Replica: 17f9285e02714fc6966f38ee2b127c3e, State: Running, Role: LEADER
I20260929 02:18:21.834403 8462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } }
I20260929 02:18:21.834929 8457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:21.836988 8464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 17f9285e02714fc6966f38ee2b127c3e. Latest consensus state: current_term: 1 leader_uuid: "17f9285e02714fc6966f38ee2b127c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } } }
I20260929 02:18:21.836982 8463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "17f9285e02714fc6966f38ee2b127c3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "17f9285e02714fc6966f38ee2b127c3e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44307 } } }
I20260929 02:18:21.837368 8463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:21.837368 8464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:21.838732 8468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:21.843717 8468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:21.852671 8468 catalog_manager.cc:1347] Generated new cluster ID: 1e0bdc55dab84e368e6885df4b86a5dd
I20260929 02:18:21.852775 8468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:21.861186 8468 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:21.862502 8468 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:21.877496 8468 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 17f9285e02714fc6966f38ee2b127c3e: Generated new TSK 0
I20260929 02:18:21.878376 8468 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:21.928361 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:22.238487 8483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:22.238744 8483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:22.238868 8483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:22.246065 8483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:22.246163 8483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:22.246301 8483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:22.255666 8483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:22.257159 8483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:22.258507 8483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:22.271303 8488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:22.271275 8491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:22.271358 8489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:22.272130 8483 server_base.cc:1034] running on GCE node
I20260929 02:18:22.272694 8483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:22.273649 8483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:22.274866 8483 hybrid_clock.cc:648] HybridClock initialized: now 1790648302274789 us; error 54 us; skew 500 ppm
I20260929 02:18:22.275174 8483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:22 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648302, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:22.278126 8483 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:22.279495 8483 webserver.cc:460] Webserver started at http://127.7.170.1:41141/ using document root <none> and password file <none>
I20260929 02:18:22.280140 8483 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:22.280251 8483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:22.280529 8483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:22.283318 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "119b6272f0cb45d28dd49af1ee1e8755"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "e478f49c1541c90c43e6b45939daf3ec"
server_key_iv: "33c3bfa059b79cf76726231a6c2c9942"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.284072 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "119b6272f0cb45d28dd49af1ee1e8755"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "e478f49c1541c90c43e6b45939daf3ec"
server_key_iv: "33c3bfa059b79cf76726231a6c2c9942"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.289026 8483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 02:18:22.292263 8498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.293635 8483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:22.293781 8483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "119b6272f0cb45d28dd49af1ee1e8755"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "e478f49c1541c90c43e6b45939daf3ec"
server_key_iv: "33c3bfa059b79cf76726231a6c2c9942"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.293948 8483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:22.321529 8483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:22.325562 8483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:22.325820 8483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:22.326743 8483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:22.328037 8483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:22.328114 8483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.328236 8483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:22.328286 8483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.357234 8483 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:43561
I20260929 02:18:22.357285 8610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:43561 every 8 connection(s)
I20260929 02:18:22.358418 8483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:22.365749 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8483
I20260929 02:18:22.365972 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:22.366551 7848 external_mini_cluster.cc:1428] Setting key ce52deb63f6be32669cc9e7313f0d9c6
Sep 29 02:18:22 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648302, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:22.378450 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:22.360867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:45407 (local address 127.7.170.62:44307)
0929 02:18:22.361281 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:22.361285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:22.362246 (+ 961us) server_negotiation.cc:407] Connection header received
0929 02:18:22.364063 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:22.364069 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:22.364148 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:22.364240 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:22.365742 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.366522 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.368346 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.368974 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.369100 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:22.372361 (+ 3261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:22.372366 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:22.372370 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:22.372401 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:22.374987 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:22.375774 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:22.375779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:22.375783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:22.376069 (+ 286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:22.376594 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:22.376599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:22.376602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:22.376963 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:22.377124 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:22.377845 (+ 721us) server_negotiation.cc:299] Negotiation successful
0929 02:18:22.378069 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":224,"threads_started":1}
I20260929 02:18:22.379392 8614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:22.361380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44307 (local address 127.7.170.1:45407)
0929 02:18:22.362086 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:22.362123 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:22.363819 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:22.364477 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:22.364485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:22.365010 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:22.365593 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:22.365610 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.366656 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.366660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:22.368219 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:22.368232 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.369172 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.369177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:22.369369 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:22.370035 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:22.370058 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:22.372146 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:22.375111 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:22.375120 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:22.375136 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:22.375643 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:22.376206 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:22.376211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:22.376215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:22.376467 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:22.377088 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:22.377096 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:22.377424 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:18:22.377731 (+ 307us) client_negotiation.cc:240] Negotiation successful
0929 02:18:22.378209 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":260,"threads_started":1}
I20260929 02:18:22.380873 8611 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44307
I20260929 02:18:22.381220 8611 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:22.382079 8611 heartbeater.cc:511] Master 127.7.170.62:44307 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:22.384279 8422 ts_manager.cc:194] Registered new tserver with Master: 119b6272f0cb45d28dd49af1ee1e8755 (127.7.170.1:43561)
I20260929 02:18:22.385740 8422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:45407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:22.403749 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:22.709215 8617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:22.709493 8617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:22.709616 8617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:22.716645 8617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:22.716750 8617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:22.716895 8617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:22.726579 8617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:22.727814 8617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:22.729121 8617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:22.743077 8622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:22.743099 8623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:22.743148 8625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:22.743988 8617 server_base.cc:1034] running on GCE node
I20260929 02:18:22.744555 8617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:22.745503 8617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:22.746690 8617 hybrid_clock.cc:648] HybridClock initialized: now 1790648302746634 us; error 48 us; skew 500 ppm
I20260929 02:18:22.746963 8617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:22 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648302, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:22.750003 8617 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:22.751418 8617 webserver.cc:460] Webserver started at http://127.7.170.2:46063/ using document root <none> and password file <none>
I20260929 02:18:22.752099 8617 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:22.752216 8617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:22.752503 8617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:22.755358 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "a53e4c3eeddc41ff98137548a22d06da"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "451243d29fbbd6b147234959b4368328"
server_key_iv: "b0a228f976c609dcd386692174001030"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.756124 8617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "a53e4c3eeddc41ff98137548a22d06da"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "451243d29fbbd6b147234959b4368328"
server_key_iv: "b0a228f976c609dcd386692174001030"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.761157 8617 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:18:22.764703 8632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.766099 8617 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:22.766244 8617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "a53e4c3eeddc41ff98137548a22d06da"
format_stamp: "Formatted at 2026-09-29 02:18:22 on dist-test-slave-8cs2"
server_key: "451243d29fbbd6b147234959b4368328"
server_key_iv: "b0a228f976c609dcd386692174001030"
server_key_version: "encryptionkey@0"
I20260929 02:18:22.766436 8617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:22.791520 8617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:22.795339 8617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:22.795567 8617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:22.796507 8617 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:22.798238 8617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:22.798326 8617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.798439 8617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:22.798492 8617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:22.827580 8617 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:35395
I20260929 02:18:22.827647 8744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:35395 every 8 connection(s)
I20260929 02:18:22.828812 8617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:22.838874 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8617
I20260929 02:18:22.839059 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:22.839468 7848 external_mini_cluster.cc:1428] Setting key 6f3869f8b591fc9b6d0963739e1ca902
Sep 29 02:18:22 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648302, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:22.847877 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:22.831330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:59097 (local address 127.7.170.62:44307)
0929 02:18:22.831509 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:22.831517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:22.832712 (+ 1195us) server_negotiation.cc:407] Connection header received
0929 02:18:22.834515 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:22.834520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:22.834598 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:22.834713 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:22.836222 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.837039 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.838571 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.838991 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.839103 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:22.842603 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:22.842608 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:22.842611 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:22.842641 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:22.845097 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:22.845745 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:22.845751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:22.845754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:22.845899 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:22.846298 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:22.846302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:22.846304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:22.846551 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:22.846672 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:22.847326 (+ 654us) server_negotiation.cc:299] Negotiation successful
0929 02:18:22.847544 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:18:22.848804 8747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:22.831833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44307 (local address 127.7.170.2:59097)
0929 02:18:22.832543 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:22.832587 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:22.834258 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:22.834948 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:22.834957 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:22.835480 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:22.836070 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:22.836088 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.837181 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.837185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:22.838415 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:22.838427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:22.839201 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:22.839208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:22.839423 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:22.840087 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:22.840110 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:22.842415 (+ 2305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:22.845220 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:22.845229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:22.845245 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:22.845640 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:22.846005 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:22.846010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:22.846014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:22.846189 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:22.846673 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:22.846680 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:22.846984 (+ 304us) client_negotiation.cc:769] Sending connection context
0929 02:18:22.847221 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:18:22.847713 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":273,"threads_started":1}
I20260929 02:18:22.850273 8745 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44307
I20260929 02:18:22.850550 8745 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:22.851224 8745 heartbeater.cc:511] Master 127.7.170.62:44307 requested a full tablet report, sending...
I20260929 02:18:22.852591 8422 ts_manager.cc:194] Registered new tserver with Master: a53e4c3eeddc41ff98137548a22d06da (127.7.170.2:35395)
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:22.853386 8422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:59097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:22.875355 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:23.176507 8750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:23.176767 8750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:23.176863 8750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:23.183715 8750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:23.183808 8750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:23.183950 8750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:23.193287 8750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44307
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:23.194453 8750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:23.195684 8750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:23.208155 8755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:23.208122 8756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:23.208410 8758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:23.209080 8750 server_base.cc:1034] running on GCE node
I20260929 02:18:23.209617 8750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:23.210493 8750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:23.211681 8750 hybrid_clock.cc:648] HybridClock initialized: now 1790648303211636 us; error 34 us; skew 500 ppm
I20260929 02:18:23.211963 8750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:23.214845 8750 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:23.216204 8750 webserver.cc:460] Webserver started at http://127.7.170.3:37367/ using document root <none> and password file <none>
I20260929 02:18:23.216833 8750 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:23.216988 8750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:23.217270 8750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:23.219986 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "c7ea5d3cd4484fcfbf358762542c16e2"
format_stamp: "Formatted at 2026-09-29 02:18:23 on dist-test-slave-8cs2"
server_key: "2a848e9ed6bfdd229896354e90294018"
server_key_iv: "744cdb12298b9d96dcd9ddc4342c3d1a"
server_key_version: "encryptionkey@0"
I20260929 02:18:23.220697 8750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "c7ea5d3cd4484fcfbf358762542c16e2"
format_stamp: "Formatted at 2026-09-29 02:18:23 on dist-test-slave-8cs2"
server_key: "2a848e9ed6bfdd229896354e90294018"
server_key_iv: "744cdb12298b9d96dcd9ddc4342c3d1a"
server_key_version: "encryptionkey@0"
I20260929 02:18:23.225484 8750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:18:23.229283 8765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:23.231271 8750 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:18:23.231434 8750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "c7ea5d3cd4484fcfbf358762542c16e2"
format_stamp: "Formatted at 2026-09-29 02:18:23 on dist-test-slave-8cs2"
server_key: "2a848e9ed6bfdd229896354e90294018"
server_key_iv: "744cdb12298b9d96dcd9ddc4342c3d1a"
server_key_version: "encryptionkey@0"
I20260929 02:18:23.231612 8750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:23.249500 8750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:23.253520 8750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:23.253789 8750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:23.254688 8750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:23.256057 8750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:23.256189 8750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:23.256358 8750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:23.256417 8750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:23.285612 8750 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:44603
I20260929 02:18:23.285672 8877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:44603 every 8 connection(s)
I20260929 02:18:23.286856 8750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:23.289517 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8750
I20260929 02:18:23.289714 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:23.290236 7848 external_mini_cluster.cc:1428] Setting key 00aea4b4fc95f708b2bc1f64ba036a32
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:23.306982 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.289340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:34947 (local address 127.7.170.62:44307)
0929 02:18:23.289491 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.289495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.291040 (+ 1545us) server_negotiation.cc:407] Connection header received
0929 02:18:23.293338 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.293343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.293430 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.293510 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.294921 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.295621 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.297211 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.297665 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.297772 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.300988 (+ 3216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.300995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.301000 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.301030 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.303926 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.304701 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.304707 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.304710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.304847 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.305278 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.305282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.305286 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.305593 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.305699 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.306327 (+ 628us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.306726 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:18:23.307768 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.290074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44307 (local address 127.7.170.3:34947)
0929 02:18:23.290884 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.290932 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.293086 (+ 2154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.293749 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.293757 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.294233 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:23.294762 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.294779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.295765 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.295770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.297082 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.297094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.297883 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.297890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.298118 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.298736 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:23.298761 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:23.300761 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:23.304068 (+ 3307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:23.304076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:23.304092 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:23.304594 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:23.304982 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:23.304987 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:23.304991 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:23.305180 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:23.305697 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:23.305703 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:23.306033 (+ 330us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.306251 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.306628 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":307,"threads_started":1}
I20260929 02:18:23.309000 8878 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44307
I20260929 02:18:23.309266 8878 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:23.309952 8878 heartbeater.cc:511] Master 127.7.170.62:44307 requested a full tablet report, sending...
I20260929 02:18:23.311322 8422 ts_manager.cc:194] Registered new tserver with Master: c7ea5d3cd4484fcfbf358762542c16e2 (127.7.170.3:44603)
I20260929 02:18:23.312204 8422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:34947
I20260929 02:18:23.317529 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:18:23.336397 8613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.323850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.7.170.62:44307)
0929 02:18:23.324027 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.324032 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.324446 (+ 414us) server_negotiation.cc:407] Connection header received
0929 02:18:23.324622 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.324626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.324688 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.324774 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.326358 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.327077 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.328426 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.328863 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.328977 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.331369 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.331375 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.331378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.331411 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.333986 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.334549 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.334553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.334555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.334674 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.335033 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.335036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.335038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.335345 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.335541 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.335918 (+ 377us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.336112 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:18:23.342561 8422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:23.349606 8422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:23.388617 8611 heartbeater.cc:503] Master 127.7.170.62:44307 was elected leader, sending a full tablet report...
I20260929 02:18:23.395877 8890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.361275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:35395 (local address 127.0.0.1:54150)
0929 02:18:23.387354 (+ 26079us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.387378 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.387477 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.388080 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.388086 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.388121 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:23.388493 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.388506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.390820 (+ 2314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.390825 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.393363 (+ 2538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.393376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.394982 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.394989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.395156 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.395198 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.395305 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.395502 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25943,"mutex_wait_us":3,"thread_start_us":27877,"threads_started":1}
I20260929 02:18:23.395843 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.361500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.7.170.2:35395)
0929 02:18:23.385711 (+ 24211us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.385720 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.387486 (+ 1766us) server_negotiation.cc:407] Connection header received
0929 02:18:23.387631 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.387639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.387881 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.388055 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:23.388686 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.390659 (+ 1973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.393526 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.394793 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.394925 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.395133 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.395357 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.395516 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24057,"thread_start_us":26824,"threads_started":1}
I20260929 02:18:23.397068 8894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.361144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:43561 (local address 127.0.0.1:56206)
0929 02:18:23.387010 (+ 25866us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.387064 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.387267 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.388302 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.388306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.388330 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:23.389520 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.389533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.391866 (+ 2333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.391872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.394266 (+ 2394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.394280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.396306 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.396312 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.396490 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.396520 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.396601 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.396721 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25615,"thread_start_us":25949,"threads_started":1}
I20260929 02:18:23.397172 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.361406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56206 (local address 127.7.170.1:43561)
0929 02:18:23.386151 (+ 24745us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.386160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.387391 (+ 1231us) server_negotiation.cc:407] Connection header received
0929 02:18:23.387522 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.387533 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.387900 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.388114 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:23.390288 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.391728 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.394436 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.395817 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.395944 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.396167 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.396684 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.396838 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24597,"thread_start_us":24483,"threads_started":1}
I20260929 02:18:23.400025 8680 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6a416f690b47fd9cc0bb22730b36eb (DEFAULT_TABLE table=test-table [id=fb06a47e6b724e6b8bb16d50fa78b022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:23.400645 8546 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6a416f690b47fd9cc0bb22730b36eb (DEFAULT_TABLE table=test-table [id=fb06a47e6b724e6b8bb16d50fa78b022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:23.401355 8680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6a416f690b47fd9cc0bb22730b36eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:23.401496 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.363280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:44603 (local address 127.0.0.1:49966)
0929 02:18:23.389926 (+ 26646us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.389943 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.390131 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.390725 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.390730 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.390752 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:23.391266 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.391277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.393065 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.393071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.394848 (+ 1777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.394874 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.396834 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.396840 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.397030 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.397065 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.397148 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.401165 (+ 4017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26508,"thread_start_us":23022,"threads_started":1}
I20260929 02:18:23.402225 8546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6a416f690b47fd9cc0bb22730b36eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:23.402513 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.364074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.7.170.3:44603)
0929 02:18:23.385293 (+ 21219us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.385302 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.390040 (+ 4738us) server_negotiation.cc:407] Connection header received
0929 02:18:23.390316 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.390324 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.390562 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.390727 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:23.391434 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.392910 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.395036 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.396371 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.396485 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.396702 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.401756 (+ 5054us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.401940 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20755,"thread_start_us":20863,"threads_started":1}
I20260929 02:18:23.406378 8813 tablet_service.cc:1467] Processing CreateTablet for tablet 0c6a416f690b47fd9cc0bb22730b36eb (DEFAULT_TABLE table=test-table [id=fb06a47e6b724e6b8bb16d50fa78b022]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:23.407842 8813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c6a416f690b47fd9cc0bb22730b36eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:23.414149 8896 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Bootstrap starting.
I20260929 02:18:23.417296 8896 tablet_bootstrap.cc:654] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:23.417616 8897 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Bootstrap starting.
I20260929 02:18:23.418682 8896 log.cc:824] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:23.421595 8897 tablet_bootstrap.cc:654] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:23.421742 8896 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: No bootstrap required, opened a new log
I20260929 02:18:23.421953 8899 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Bootstrap starting.
I20260929 02:18:23.422006 8896 ts_tablet_manager.cc:1397] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:18:23.422787 8897 log.cc:824] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:23.425057 8899 tablet_bootstrap.cc:654] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:23.425701 8897 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: No bootstrap required, opened a new log
I20260929 02:18:23.425990 8897 ts_tablet_manager.cc:1397] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260929 02:18:23.426127 8899 log.cc:824] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:23.428822 8899 tablet_bootstrap.cc:492] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: No bootstrap required, opened a new log
I20260929 02:18:23.429080 8899 ts_tablet_manager.cc:1397] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260929 02:18:23.431929 8896 raft_consensus.cc:348] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.432382 8896 raft_consensus.cc:374] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:23.432611 8896 raft_consensus.cc:729] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 119b6272f0cb45d28dd49af1ee1e8755, State: Initialized, Role: FOLLOWER
I20260929 02:18:23.433279 8896 consensus_queue.cc:260] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.435074 8896 ts_tablet_manager.cc:1428] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260929 02:18:23.436386 8897 raft_consensus.cc:348] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.436717 8897 raft_consensus.cc:374] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:23.436860 8897 raft_consensus.cc:729] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53e4c3eeddc41ff98137548a22d06da, State: Initialized, Role: FOLLOWER
I20260929 02:18:23.437407 8897 consensus_queue.cc:260] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.437886 8899 raft_consensus.cc:348] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.438277 8899 raft_consensus.cc:374] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:23.438376 8899 raft_consensus.cc:729] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ea5d3cd4484fcfbf358762542c16e2, State: Initialized, Role: FOLLOWER
I20260929 02:18:23.438493 8745 heartbeater.cc:503] Master 127.7.170.62:44307 was elected leader, sending a full tablet report...
I20260929 02:18:23.438897 8897 ts_tablet_manager.cc:1428] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260929 02:18:23.439033 8899 consensus_queue.cc:260] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.440609 8899 ts_tablet_manager.cc:1428] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260929 02:18:23.441275 8878 heartbeater.cc:503] Master 127.7.170.62:44307 was elected leader, sending a full tablet report...
I20260929 02:18:23.522333 8904 raft_consensus.cc:482] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:23.522616 8904 raft_consensus.cc:504] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.524681 8904 leader_election.cc:288] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 119b6272f0cb45d28dd49af1ee1e8755 (127.7.170.1:43561), a53e4c3eeddc41ff98137548a22d06da (127.7.170.2:35395)
I20260929 02:18:23.530071 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.525078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:57993 (local address 127.7.170.1:43561)
0929 02:18:23.525260 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.525265 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.525283 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:18:23.525450 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.525455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.525529 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.525620 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:23.526112 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.526830 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.528409 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.529268 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.529393 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.529498 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.529730 (+ 232us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.529848 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:18:23.530092 8880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.524843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:43561 (local address 127.7.170.3:57993)
0929 02:18:23.525140 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.525161 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.525282 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.525678 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.525683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.525724 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:23.525956 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.525966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.526970 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.526974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.528254 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.528269 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.529415 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.529420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.529583 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.529613 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.529690 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.529812 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":51}
I20260929 02:18:23.530848 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.525211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41763 (local address 127.7.170.2:35395)
0929 02:18:23.525400 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.525407 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.526013 (+ 606us) server_negotiation.cc:407] Connection header received
0929 02:18:23.526151 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.526155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.526211 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.526279 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:23.526743 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.527483 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.529238 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.530011 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.530129 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.530223 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.530502 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.530621 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:18:23.530982 8905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.525116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:35395 (local address 127.7.170.3:41763)
0929 02:18:23.525916 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:23.525929 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:23.526032 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:23.526307 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:23.526311 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:23.526340 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:23.526597 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.526605 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.527623 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.527626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.529084 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:23.529096 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.530207 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.530213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:23.530370 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.530399 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:18:23.530479 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:18:23.530593 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":1,"thread_start_us":447,"threads_started":1}
I20260929 02:18:23.531075 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6a416f690b47fd9cc0bb22730b36eb" candidate_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119b6272f0cb45d28dd49af1ee1e8755" is_pre_election: true
I20260929 02:18:23.531613 8566 raft_consensus.cc:2454] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ea5d3cd4484fcfbf358762542c16e2 in term 0.
I20260929 02:18:23.531859 8700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6a416f690b47fd9cc0bb22730b36eb" candidate_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53e4c3eeddc41ff98137548a22d06da" is_pre_election: true
I20260929 02:18:23.532496 8767 leader_election.cc:302] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119b6272f0cb45d28dd49af1ee1e8755, c7ea5d3cd4484fcfbf358762542c16e2; no voters:
I20260929 02:18:23.532873 8700 raft_consensus.cc:2454] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ea5d3cd4484fcfbf358762542c16e2 in term 0.
I20260929 02:18:23.532974 8904 raft_consensus.cc:2781] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:23.533092 8904 raft_consensus.cc:482] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:23.533177 8904 raft_consensus.cc:3037] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:23.535555 8904 raft_consensus.cc:504] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.536134 8904 leader_election.cc:288] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [CANDIDATE]: Term 1 election: Requested vote from peers 119b6272f0cb45d28dd49af1ee1e8755 (127.7.170.1:43561), a53e4c3eeddc41ff98137548a22d06da (127.7.170.2:35395)
I20260929 02:18:23.536607 8700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6a416f690b47fd9cc0bb22730b36eb" candidate_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53e4c3eeddc41ff98137548a22d06da"
I20260929 02:18:23.536610 8566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c6a416f690b47fd9cc0bb22730b36eb" candidate_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "119b6272f0cb45d28dd49af1ee1e8755"
I20260929 02:18:23.536787 8566 raft_consensus.cc:3037] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:23.536775 8700 raft_consensus.cc:3037] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:18:23.538391 8879 tablet.cc:2369] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:23.539201 8566 raft_consensus.cc:2454] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ea5d3cd4484fcfbf358762542c16e2 in term 1.
I20260929 02:18:23.539372 8700 raft_consensus.cc:2454] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ea5d3cd4484fcfbf358762542c16e2 in term 1.
I20260929 02:18:23.539551 8767 leader_election.cc:302] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 119b6272f0cb45d28dd49af1ee1e8755, c7ea5d3cd4484fcfbf358762542c16e2; no voters:
I20260929 02:18:23.539812 8904 raft_consensus.cc:2781] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:23.540241 8904 raft_consensus.cc:686] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [term 1 LEADER]: Becoming Leader. State: Replica: c7ea5d3cd4484fcfbf358762542c16e2, State: Running, Role: LEADER
I20260929 02:18:23.540661 8904 consensus_queue.cc:237] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } }
I20260929 02:18:23.545296 8422 catalog_manager.cc:5495] T 0c6a416f690b47fd9cc0bb22730b36eb P c7ea5d3cd4484fcfbf358762542c16e2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7ea5d3cd4484fcfbf358762542c16e2 (127.7.170.3). New cstate: current_term: 1 leader_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "119b6272f0cb45d28dd49af1ee1e8755" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a53e4c3eeddc41ff98137548a22d06da" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 35395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ea5d3cd4484fcfbf358762542c16e2" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44603 } health_report { overall_health: HEALTHY } } }
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:23.569993 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.554617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.7.170.1:43561)
0929 02:18:23.554779 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.554783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.555633 (+ 850us) server_negotiation.cc:407] Connection header received
0929 02:18:23.555797 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.555801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.555882 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.555961 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.557764 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.558576 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.559777 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.560192 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.560295 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.564196 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.564211 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.564232 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.564273 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.567229 (+ 2956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.567905 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.567914 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.567916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.568053 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.568439 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.568443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.568446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.568797 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.569061 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.569542 (+ 481us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.569701 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:18:23.580775 8746 tablet.cc:2369] T 0c6a416f690b47fd9cc0bb22730b36eb P a53e4c3eeddc41ff98137548a22d06da: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
W20260929 02:18:23.610579 8612 tablet.cc:2369] T 0c6a416f690b47fd9cc0bb22730b36eb P 119b6272f0cb45d28dd49af1ee1e8755: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:23.611958 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.598597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56224 (local address 127.7.170.1:43561)
0929 02:18:23.598833 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.598839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.599111 (+ 272us) server_negotiation.cc:407] Connection header received
0929 02:18:23.599268 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.599273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.599336 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.599428 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.600646 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.601403 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.602656 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.603097 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.603202 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.607031 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.607036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.607040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.607074 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.609618 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.610165 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.610168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.610171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.610281 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.610744 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.610750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.610753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.611038 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.611215 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.611497 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.611658 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:23.653019 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.640047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56226 (local address 127.7.170.1:43561)
0929 02:18:23.640227 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.640233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.640557 (+ 324us) server_negotiation.cc:407] Connection header received
0929 02:18:23.640723 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.640727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.640789 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.640878 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.642272 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.642950 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.644122 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.644533 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.644637 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.648239 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.648244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.648247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.648278 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.650604 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.651230 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.651236 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.651239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.651355 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.651684 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.651688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.651692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.651954 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.652182 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.652458 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.652671 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 02:18:23.654596 8526 tablet_service.cc:2908] Rejecting scan request for tablet 0c6a416f690b47fd9cc0bb22730b36eb: Uninitialized: clean time has not yet been initialized
I20260929 02:18:23.657359 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8483
I20260929 02:18:23.668779 8892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.658134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56232 (local address 127.7.170.1:43561)
0929 02:18:23.658268 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.658272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.658585 (+ 313us) server_negotiation.cc:407] Connection header received
0929 02:18:23.658695 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.658698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.658746 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.658815 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.659825 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.660488 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.661598 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.661985 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.662099 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.664006 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.664011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.664014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.664041 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.666310 (+ 2269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.666838 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.666841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.666843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.667327 (+ 484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.667613 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.667616 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.667618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.667859 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.668082 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.668390 (+ 308us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.668557 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:18:23.669446 8606 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:23.755626 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8483
I20260929 02:18:23.769722 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8617
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:23.783530 8893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.770448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.7.170.2:35395)
0929 02:18:23.770628 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.770632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.770652 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:18:23.770787 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.770790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.770874 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.770990 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.772075 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.772820 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.774105 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.774482 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.774571 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.778279 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.778293 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.778308 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.778341 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.781038 (+ 2697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.781641 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.781649 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.781651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.781772 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.782096 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.782100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.782102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.782474 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.782768 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.783016 (+ 248us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.783208 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:18:23.784169 8740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:23.869725 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8617
I20260929 02:18:23.883059 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8750
Sep 29 02:18:23 dist-test-slave-8cs2 krb5kdc[8377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:23.896611 8895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:23.883868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.7.170.3:44603)
0929 02:18:23.884062 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:23.884068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:23.884088 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:18:23.884268 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:23.884271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:23.884348 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:23.884444 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:23.885631 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.886345 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.887581 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:23.887985 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:23.888090 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:23.891892 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:23.891906 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:23.891922 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:23.891958 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:23.894453 (+ 2495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.895044 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.895051 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.895054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.895170 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:23.895433 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:23.895435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:23.895438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:23.895748 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:23.895953 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:23.896178 (+ 225us) server_negotiation.cc:299] Negotiation successful
0929 02:18:23.896335 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260929 02:18:23.897289 8873 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:23.983408 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8750
I20260929 02:18:23.996405 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8390
I20260929 02:18:23.997460 8452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:24.102170 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8390
2026-09-29T02:18:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (3859 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:24 dist-test-slave-8cs2 krb5kdc[8925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:24 dist-test-slave-8cs2 krb5kdc[8925](info): set up 2 sockets
Sep 29 02:18:24 dist-test-slave-8cs2 krb5kdc[8925](info): commencing operation
krb5kdc: starting...
W20260929 02:18:25.100962 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.000s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:25 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:25.264422 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39439
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:39439
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:25.563810 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:25.564103 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:25.564232 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:25.571372 8938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:25.571482 8938 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:18:25.571547 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:25.571626 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:25.571687 8938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:25.571765 8938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:25.581473 8938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:39439
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:25.582731 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:25.584067 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:25.595438 8943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:25.595402 8944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:25.595456 8946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:25.596407 8938 server_base.cc:1034] running on GCE node
I20260929 02:18:25.597237 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:25.598546 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:25.599781 8938 hybrid_clock.cc:648] HybridClock initialized: now 1790648305599709 us; error 38 us; skew 500 ppm
I20260929 02:18:25.600095 8938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:25 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:25.602999 8938 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:25.604410 8938 webserver.cc:460] Webserver started at http://127.7.170.62:45115/ using document root <none> and password file <none>
I20260929 02:18:25.605085 8938 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:25.605180 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:25.605484 8938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:25.608265 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "a84924cf52414b4e921fa77b0eee425c"
format_stamp: "Formatted at 2026-09-29 02:18:25 on dist-test-slave-8cs2"
server_key: "c60983c7b1fd0d8475ff463c964550e2"
server_key_iv: "c76f22e5c26bd9eade27abb20830fa35"
server_key_version: "encryptionkey@0"
I20260929 02:18:25.609040 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "a84924cf52414b4e921fa77b0eee425c"
format_stamp: "Formatted at 2026-09-29 02:18:25 on dist-test-slave-8cs2"
server_key: "c60983c7b1fd0d8475ff463c964550e2"
server_key_iv: "c76f22e5c26bd9eade27abb20830fa35"
server_key_version: "encryptionkey@0"
I20260929 02:18:25.613888 8938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:18:25.617159 8953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:25.618474 8938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:25.618633 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "a84924cf52414b4e921fa77b0eee425c"
format_stamp: "Formatted at 2026-09-29 02:18:25 on dist-test-slave-8cs2"
server_key: "c60983c7b1fd0d8475ff463c964550e2"
server_key_iv: "c76f22e5c26bd9eade27abb20830fa35"
server_key_version: "encryptionkey@0"
I20260929 02:18:25.618834 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:25.641403 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:25.645299 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:25.645543 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:25.664232 8938 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:39439
I20260929 02:18:25.664235 9014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:39439 every 8 connection(s)
I20260929 02:18:25.665580 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:25.669263 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 8938
I20260929 02:18:25.669438 9015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:25.669526 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:25.670300 7848 external_mini_cluster.cc:1428] Setting key ec23a9ed9bd727ae5fd56c16bc6f7ac8
I20260929 02:18:25.682964 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Bootstrap starting.
Sep 29 02:18:25 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:25.686663 9015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:25.687770 9015 log.cc:824] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:25.690874 9015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: No bootstrap required, opened a new log
I20260929 02:18:25.693387 9018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:25.672433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.7.170.62:39439)
0929 02:18:25.673307 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:25.673319 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:25.673358 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:18:25.675576 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:25.675603 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:25.676027 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:25.676369 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:25.678016 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:25.679262 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:25.681009 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:25.681668 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:25.681802 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:25.686350 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:25.686358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:25.686372 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:25.686420 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:25.689530 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:25.690270 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:25.690275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:25.690281 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:25.690445 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:25.690796 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:25.690799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:25.690801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:25.691312 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:25.691528 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:25.691961 (+ 433us) server_negotiation.cc:299] Negotiation successful
0929 02:18:25.692330 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":452,"threads_started":1}
I20260929 02:18:25.699067 9015 raft_consensus.cc:348] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } }
I20260929 02:18:25.699385 9015 raft_consensus.cc:374] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:25.699483 9015 raft_consensus.cc:729] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a84924cf52414b4e921fa77b0eee425c, State: Initialized, Role: FOLLOWER
I20260929 02:18:25.699990 9015 consensus_queue.cc:260] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } }
I20260929 02:18:25.700192 9015 raft_consensus.cc:388] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:25.700310 9015 raft_consensus.cc:482] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:25.700446 9015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:25.702641 9015 raft_consensus.cc:504] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } }
I20260929 02:18:25.703094 9015 leader_election.cc:302] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a84924cf52414b4e921fa77b0eee425c; no voters:
I20260929 02:18:25.703588 9015 leader_election.cc:288] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:25.703691 9020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:25.704304 9020 raft_consensus.cc:686] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [term 1 LEADER]: Becoming Leader. State: Replica: a84924cf52414b4e921fa77b0eee425c, State: Running, Role: LEADER
I20260929 02:18:25.704834 9020 consensus_queue.cc:237] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } }
I20260929 02:18:25.705493 9015 sys_catalog.cc:564] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:25.707569 9021 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a84924cf52414b4e921fa77b0eee425c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } } }
I20260929 02:18:25.707614 9022 sys_catalog.cc:455] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a84924cf52414b4e921fa77b0eee425c. Latest consensus state: current_term: 1 leader_uuid: "a84924cf52414b4e921fa77b0eee425c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a84924cf52414b4e921fa77b0eee425c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39439 } } }
I20260929 02:18:25.708014 9022 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:25.708014 9021 sys_catalog.cc:458] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:25.711510 9028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:25.717311 9028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:25.726447 9028 catalog_manager.cc:1347] Generated new cluster ID: 64a8f907089b464791b9d8f7dc1a1012
I20260929 02:18:25.726576 9028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:25.734439 9028 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:25.735777 9028 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:25.745146 9028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Generated new TSK 0
I20260929 02:18:25.745975 9028 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:25.800266 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39439
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:26.128862 9041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:26.129168 9041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:26.129317 9041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:26.136253 9041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:26.136356 9041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:26.136478 9041 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:26.136574 9041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:26.145982 9041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:39439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:26.147207 9041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:26.148530 9041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:26.160713 9046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:26.160650 9047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:26.160691 9049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:26.161768 9041 server_base.cc:1034] running on GCE node
I20260929 02:18:26.162340 9041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:26.163164 9041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:26.164379 9041 hybrid_clock.cc:648] HybridClock initialized: now 1790648306164321 us; error 31 us; skew 500 ppm
I20260929 02:18:26.164652 9041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:26 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:26.167558 9041 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:26.168905 9041 webserver.cc:460] Webserver started at http://127.7.170.1:34213/ using document root <none> and password file <none>
I20260929 02:18:26.169575 9041 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:26.169668 9041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:26.169962 9041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:26.172668 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "b7ff696dccef4b59abe921d1f721bf40"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "bf90ace44705e43e50b54a4b703e753d"
server_key_iv: "79a769b546b9e3bdb09c4ab667915d70"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.173447 9041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "b7ff696dccef4b59abe921d1f721bf40"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "bf90ace44705e43e50b54a4b703e753d"
server_key_iv: "79a769b546b9e3bdb09c4ab667915d70"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.178085 9041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:18:26.181293 9056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:26.182546 9041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:18:26.182701 9041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "b7ff696dccef4b59abe921d1f721bf40"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "bf90ace44705e43e50b54a4b703e753d"
server_key_iv: "79a769b546b9e3bdb09c4ab667915d70"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.182924 9041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:26.196615 9041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:26.200155 9041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:26.200376 9041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:26.202904 9041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:26.203011 9041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:26.203167 9041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:26.203301 9041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:26 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:26.223572 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.203284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:38159 (local address 127.7.170.62:39439)
0929 02:18:26.203726 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:26.203733 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:26.204861 (+ 1128us) server_negotiation.cc:407] Connection header received
0929 02:18:26.206971 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:26.206977 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:26.207056 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:26.207149 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:26.209211 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.210284 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.212655 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.213168 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.213285 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.217133 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:26.217138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:26.217141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:26.217172 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:26.220133 (+ 2961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.221072 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.221078 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.221081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.221239 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.221713 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.221718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.221722 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.222073 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:26.222238 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:26.222984 (+ 746us) server_negotiation.cc:299] Negotiation successful
0929 02:18:26.223199 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":218,"threads_started":1}
I20260929 02:18:26.224550 9066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.203939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.1:38159)
0929 02:18:26.204710 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:26.204754 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:26.206669 (+ 1915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:26.207411 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:26.207426 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:26.208176 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:26.208991 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.209020 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.210435 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.210440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.212490 (+ 2050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.212501 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.213572 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.213584 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.213854 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.214523 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:26.214545 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:26.216879 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:26.220243 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.220255 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.220274 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.220913 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.221366 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.221370 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.221374 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.221594 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.222219 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:26.222226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:26.222617 (+ 391us) client_negotiation.cc:769] Sending connection context
0929 02:18:26.222865 (+ 248us) client_negotiation.cc:240] Negotiation successful
0929 02:18:26.223373 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":330,"threads_started":1}
I20260929 02:18:26.243947 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.230260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:42841 (local address 127.7.170.62:39439)
0929 02:18:26.230415 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:26.230420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:26.231024 (+ 604us) server_negotiation.cc:407] Connection header received
0929 02:18:26.231133 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:26.231139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:26.231222 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:26.231324 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:26.232086 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.233405 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.235664 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.236183 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.236300 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.238641 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:26.238645 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:26.238648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:26.238677 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:26.241553 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.242156 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.242162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.242165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.242311 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.242695 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.242698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.242700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.242939 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:26.243044 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:26.243561 (+ 517us) server_negotiation.cc:299] Negotiation successful
0929 02:18:26.243730 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:18:26.244290 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.230311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.1:42841)
0929 02:18:26.230924 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:26.230943 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:26.231050 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:26.231331 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:26.231333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:26.231695 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:26.231957 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.231979 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.233580 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.233585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.235516 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.235530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.236539 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.236547 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.236716 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.237424 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:26.237443 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:26.238469 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:26.241662 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.241668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.241672 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.242052 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.242438 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.242443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.242446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.242599 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.243050 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:26.243055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:26.243246 (+ 191us) client_negotiation.cc:769] Sending connection context
0929 02:18:26.243464 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 02:18:26.243825 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":354,"threads_started":1}
I20260929 02:18:26.254801 9041 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:45481
I20260929 02:18:26.254855 9176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:45481 every 8 connection(s)
I20260929 02:18:26.255988 9041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:26.257792 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9041
I20260929 02:18:26.257859 9177 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39439
I20260929 02:18:26.258025 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:26.258064 9177 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:26.258790 7848 external_mini_cluster.cc:1428] Setting key 95ba86ce6d2fce147a9f60615a145f17
I20260929 02:18:26.258795 9177 heartbeater.cc:511] Master 127.7.170.62:39439 requested a full tablet report, sending...
I20260929 02:18:26.260843 8969 ts_manager.cc:194] Registered new tserver with Master: b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481)
I20260929 02:18:26.262243 8969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:38159
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:26.295421 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39439
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:26.597976 9181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:26.598263 9181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:26.598371 9181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:26.605461 9181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:26.605564 9181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:26.605657 9181 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:26.605757 9181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:26.615921 9181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:39439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:26.617192 9181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:26.618481 9181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:26.630952 9187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:26.630999 9189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:26.630921 9186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:26.631922 9181 server_base.cc:1034] running on GCE node
I20260929 02:18:26.632540 9181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:26.633508 9181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:26.634747 9181 hybrid_clock.cc:648] HybridClock initialized: now 1790648306634678 us; error 35 us; skew 500 ppm
I20260929 02:18:26.635064 9181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:26 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:26.638521 9181 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:26.639829 9181 webserver.cc:460] Webserver started at http://127.7.170.2:38179/ using document root <none> and password file <none>
I20260929 02:18:26.640467 9181 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:26.640547 9181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:26.640811 9181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:26.643611 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "127279b02d4b451c8bef7ef7bf3e55b0"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "4c9b8d2b7cbfa0998281f906348065f2"
server_key_iv: "c44ed533f77cf52db94829f22c4e6c00"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.644309 9181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "127279b02d4b451c8bef7ef7bf3e55b0"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "4c9b8d2b7cbfa0998281f906348065f2"
server_key_iv: "c44ed533f77cf52db94829f22c4e6c00"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.649057 9181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.006s
I20260929 02:18:26.652245 9196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:26.653637 9181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:26.653774 9181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "127279b02d4b451c8bef7ef7bf3e55b0"
format_stamp: "Formatted at 2026-09-29 02:18:26 on dist-test-slave-8cs2"
server_key: "4c9b8d2b7cbfa0998281f906348065f2"
server_key_iv: "c44ed533f77cf52db94829f22c4e6c00"
server_key_version: "encryptionkey@0"
I20260929 02:18:26.653973 9181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:26.666590 9181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:26.670590 9181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:26.670826 9181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:26.673362 9181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:26.673444 9181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:26.673580 9181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:26.673641 9181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:26 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:26.696959 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.674067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:46419 (local address 127.7.170.62:39439)
0929 02:18:26.674242 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:26.674248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:26.675742 (+ 1494us) server_negotiation.cc:407] Connection header received
0929 02:18:26.678631 (+ 2889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:26.678639 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:26.678759 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:26.678830 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:26.681846 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.682799 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.685609 (+ 2810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.686140 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.686233 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.690969 (+ 4736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:26.690977 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:26.690982 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:26.691023 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:26.693919 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.694677 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.694681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.694683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.694808 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.695233 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.695237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.695239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.695521 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:26.695658 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:26.696367 (+ 709us) server_negotiation.cc:299] Negotiation successful
0929 02:18:26.696587 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:18:26.699380 9205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.674662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.2:46419)
0929 02:18:26.675577 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:26.675624 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:26.678254 (+ 2630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:26.679523 (+ 1269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:26.679535 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:26.680412 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:26.681610 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.681644 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.682970 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.682975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.685401 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.685474 (+ 73us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.686419 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.686427 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.686762 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.687539 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:26.687571 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:26.690633 (+ 3062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:26.694061 (+ 3428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.694070 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.694088 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.694546 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.694908 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.694913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.694917 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.695124 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.695655 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:26.695663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:26.696018 (+ 355us) client_negotiation.cc:769] Sending connection context
0929 02:18:26.696253 (+ 235us) client_negotiation.cc:240] Negotiation successful
0929 02:18:26.696815 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260929 02:18:26.717856 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.705565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:55913 (local address 127.7.170.62:39439)
0929 02:18:26.705774 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:26.705781 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:26.706151 (+ 370us) server_negotiation.cc:407] Connection header received
0929 02:18:26.706317 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:26.706323 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:26.706414 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:26.706524 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:26.707374 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.708590 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.710202 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.710627 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.710735 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.712604 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:26.712609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:26.712613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:26.712643 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:26.715189 (+ 2546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.715990 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.715994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.715997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.716123 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:26.716478 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:26.716481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:26.716483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:26.716797 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:26.716902 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:26.717439 (+ 537us) server_negotiation.cc:299] Negotiation successful
0929 02:18:26.717637 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:18:26.718259 9269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:26.705304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.2:55913)
0929 02:18:26.706068 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:26.706086 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:26.706210 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:26.706543 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:26.706546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:26.706982 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:26.707256 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.707265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.708726 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.708730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.710052 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:26.710065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:26.710796 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:26.710801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:26.710953 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:26.711572 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:26.711588 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:26.712428 (+ 840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:26.715319 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.715325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.715330 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.715882 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.716214 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:26.716219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:26.716223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:26.716389 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:26.716888 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:26.716893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:26.717151 (+ 258us) client_negotiation.cc:769] Sending connection context
0929 02:18:26.717333 (+ 182us) client_negotiation.cc:240] Negotiation successful
0929 02:18:26.717688 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":407,"threads_started":1}
I20260929 02:18:26.736539 9181 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:37999
I20260929 02:18:26.736574 9315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:37999 every 8 connection(s)
I20260929 02:18:26.738109 9181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:26.740259 9316 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39439
I20260929 02:18:26.740492 9316 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:26.741307 9316 heartbeater.cc:511] Master 127.7.170.62:39439 requested a full tablet report, sending...
I20260929 02:18:26.741647 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9181
I20260929 02:18:26.741832 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:26.742275 7848 external_mini_cluster.cc:1428] Setting key 66b1a70156958ab3a8abd32c1eaa4fd8
I20260929 02:18:26.742712 8969 ts_manager.cc:194] Registered new tserver with Master: 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999)
I20260929 02:18:26.743582 8969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:46419
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:26.777314 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39439
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:38227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:27.082058 9320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:27.082314 9320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:27.082402 9320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:27.089344 9320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:27.089450 9320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:27.089535 9320 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:27.089612 9320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:27.098982 9320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:39439
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:27.100134 9320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:27.101418 9320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:27.113464 9328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:27.113422 9326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:27.113425 9325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:27.114183 9320 server_base.cc:1034] running on GCE node
I20260929 02:18:27.114670 9320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:27.115451 9320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:27.116650 9320 hybrid_clock.cc:648] HybridClock initialized: now 1790648307116589 us; error 36 us; skew 500 ppm
I20260929 02:18:27.116979 9320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:27 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:27.119742 9320 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:27.121115 9320 webserver.cc:460] Webserver started at http://127.7.170.3:39981/ using document root <none> and password file <none>
I20260929 02:18:27.121726 9320 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:27.121814 9320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:27.122104 9320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:27.124747 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "91b0dde18e604bf5b70f6d9067bf77c7"
format_stamp: "Formatted at 2026-09-29 02:18:27 on dist-test-slave-8cs2"
server_key: "83dbd201211b26ea9d23cccfb4803c95"
server_key_iv: "e24ed4cd20fa5da29de583c414944391"
server_key_version: "encryptionkey@0"
I20260929 02:18:27.125504 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "91b0dde18e604bf5b70f6d9067bf77c7"
format_stamp: "Formatted at 2026-09-29 02:18:27 on dist-test-slave-8cs2"
server_key: "83dbd201211b26ea9d23cccfb4803c95"
server_key_iv: "e24ed4cd20fa5da29de583c414944391"
server_key_version: "encryptionkey@0"
I20260929 02:18:27.130229 9320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260929 02:18:27.133391 9335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:27.134604 9320 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:18:27.134735 9320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "91b0dde18e604bf5b70f6d9067bf77c7"
format_stamp: "Formatted at 2026-09-29 02:18:27 on dist-test-slave-8cs2"
server_key: "83dbd201211b26ea9d23cccfb4803c95"
server_key_iv: "e24ed4cd20fa5da29de583c414944391"
server_key_version: "encryptionkey@0"
I20260929 02:18:27.134939 9320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:27.157814 9320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:27.161710 9320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:27.161950 9320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:27.164528 9320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:27.164629 9320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:27.164770 9320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:27.164848 9320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:27 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:27.186012 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.165020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41161 (local address 127.7.170.62:39439)
0929 02:18:27.165197 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.165203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.166665 (+ 1462us) server_negotiation.cc:407] Connection header received
0929 02:18:27.168690 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.168695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.168775 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.168858 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:27.171023 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.172164 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.174205 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.174745 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.174868 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.179144 (+ 4276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:27.179151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:27.179156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:27.179196 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:27.182603 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.183439 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.183442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.183445 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.183578 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.184139 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.184143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.184145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.184432 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:27.184575 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.185551 (+ 976us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.185721 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:18:27.187466 9344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.165672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.3:41161)
0929 02:18:27.166522 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.166559 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.168393 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.169198 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.169211 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.169955 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:27.170809 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.170846 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.172342 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.172348 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.174070 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.174084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.174966 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.174974 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.175269 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.176001 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:27.176036 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:27.178897 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:27.182731 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.182738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.182749 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.183254 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.183678 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.183687 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.183690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.183994 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.184573 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:27.184583 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:27.185064 (+ 481us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.185402 (+ 338us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.186003 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":400,"threads_started":1}
I20260929 02:18:27.216747 9320 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:42931
I20260929 02:18:27.216784 9454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:42931 every 8 connection(s)
I20260929 02:18:27.218061 9320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:27.220474 9433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.207534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.7.170.3:34209)
0929 02:18:27.208204 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.208228 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.208390 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.208863 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.208866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.209484 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:27.209926 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.209940 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.210916 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.210921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.211935 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.211945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.212743 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.212753 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.212893 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.213671 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:27.213691 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:27.214642 (+ 951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:27.217854 (+ 3212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.217859 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.217863 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.218351 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.218696 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.218701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.218705 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.218851 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.219378 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:27.219383 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:27.219630 (+ 247us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.219779 (+ 149us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.220102 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":338,"threads_started":1}
I20260929 02:18:27.220651 9455 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39439
I20260929 02:18:27.220875 9455 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:27.221432 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.207424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:34209 (local address 127.7.170.62:39439)
0929 02:18:27.207599 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.207605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.208333 (+ 728us) server_negotiation.cc:407] Connection header received
0929 02:18:27.208529 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.208536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.208622 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.208724 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:27.210077 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.210788 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.212071 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.212563 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.212670 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.214809 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:27.214815 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:27.214819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:27.214864 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:27.217726 (+ 2862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.218455 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.218459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.218461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.218586 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.218969 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.218973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.218976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.219271 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:27.219413 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.220886 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.221125 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:18:27.221868 9455 heartbeater.cc:511] Master 127.7.170.62:39439 requested a full tablet report, sending...
I20260929 02:18:27.222397 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9320
I20260929 02:18:27.222579 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:27.223088 7848 external_mini_cluster.cc:1428] Setting key a9f1f82b0b310cc0b709e6e59eaa16bf
I20260929 02:18:27.223340 8969 ts_manager.cc:194] Registered new tserver with Master: 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931)
I20260929 02:18:27.224210 8969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:41161
I20260929 02:18:27.225956 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:18:27 dist-test-slave-8cs2 krb5kdc[8925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:18:27 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:27.265116 9177 heartbeater.cc:503] Master 127.7.170.62:39439 was elected leader, sending a full tablet report...
I20260929 02:18:27.268141 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.254650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59540 (local address 127.7.170.62:39439)
0929 02:18:27.254817 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.254823 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.255214 (+ 391us) server_negotiation.cc:407] Connection header received
0929 02:18:27.255390 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.255394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.255459 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.255542 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:27.256850 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.257708 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.259474 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.259855 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.259946 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.263415 (+ 3469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:27.263420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:27.263423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:27.263453 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:27.265822 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.266391 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.266396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.266400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.266555 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.266891 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.266896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.266899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.267163 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:27.267373 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.267624 (+ 251us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.267820 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:18:27.271826 8969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:27.278371 8969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:27.301898 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.289600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37999 (local address 127.0.0.1:54378)
0929 02:18:27.290412 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.290453 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.290645 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.295577 (+ 4932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.295585 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.295614 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.295958 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.295970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.297879 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.297885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.299658 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.299669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.300911 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.300917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.301132 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.301195 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.301319 (+ 124us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.301492 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":19,"thread_start_us":499,"threads_started":1}
I20260929 02:18:27.302749 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.289938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.7.170.2:37999)
0929 02:18:27.294789 (+ 4851us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.294802 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.294832 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:18:27.294963 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.294972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.295227 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.296141 (+ 914us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.296350 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.297772 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.299802 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.300739 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.300863 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.301087 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.302305 (+ 1218us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.302479 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4660,"thread_start_us":4539,"threads_started":1}
I20260929 02:18:27.303200 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.289996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39448 (local address 127.7.170.3:42931)
0929 02:18:27.295641 (+ 5645us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.295648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.295682 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:18:27.295766 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.295773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.296011 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.296207 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.296793 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.298270 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.300691 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.302243 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.302373 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.302625 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.302814 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.302967 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5486,"thread_start_us":5328,"threads_started":1}
I20260929 02:18:27.303419 9469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.291407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45481 (local address 127.0.0.1:53622)
0929 02:18:27.295234 (+ 3827us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.295252 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.295430 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.295994 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.295998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.296025 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.296449 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.296472 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.298140 (+ 1668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.298145 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.300417 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.300431 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.302684 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.302690 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.302839 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.302870 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.302937 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.303057 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3709,"mutex_wait_us":19,"thread_start_us":536,"threads_started":1}
I20260929 02:18:27.304069 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.291833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53622 (local address 127.7.170.1:45481)
0929 02:18:27.293390 (+ 1557us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.293399 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.295344 (+ 1945us) server_negotiation.cc:407] Connection header received
0929 02:18:27.295675 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.295681 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.295839 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.295999 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.296630 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.298031 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.300579 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.302257 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.302369 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.302560 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.303629 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.303783 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1404,"thread_start_us":1292,"threads_started":1}
I20260929 02:18:27.304277 9472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.289863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42931 (local address 127.0.0.1:39448)
0929 02:18:27.291851 (+ 1988us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.291867 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.291971 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.296202 (+ 4231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.296207 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.296226 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.296615 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.296626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.298390 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.298395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.300562 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.300573 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.302422 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.302428 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.302600 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.302634 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.302774 (+ 140us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.303971 (+ 1197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1882,"mutex_wait_us":3,"thread_start_us":5112,"threads_started":1}
I20260929 02:18:27.306560 9246 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4c6a74e7d74832870b918816fb6840 (DEFAULT_TABLE table=test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.307804 9246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4c6a74e7d74832870b918816fb6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.307345 9107 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4c6a74e7d74832870b918816fb6840 (DEFAULT_TABLE table=test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.307345 9385 tablet_service.cc:1467] Processing CreateTablet for tablet 7b4c6a74e7d74832870b918816fb6840 (DEFAULT_TABLE table=test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.308663 9385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4c6a74e7d74832870b918816fb6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.308656 9107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b4c6a74e7d74832870b918816fb6840. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.320151 9474 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Bootstrap starting.
I20260929 02:18:27.322180 9476 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Bootstrap starting.
I20260929 02:18:27.323488 9474 tablet_bootstrap.cc:654] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.324870 9474 log.cc:824] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:27.325700 9476 tablet_bootstrap.cc:654] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.326200 9475 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Bootstrap starting.
I20260929 02:18:27.326897 9476 log.cc:824] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:27.329712 9475 tablet_bootstrap.cc:654] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.330014 9476 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: No bootstrap required, opened a new log
I20260929 02:18:27.330284 9476 ts_tablet_manager.cc:1397] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 02:18:27.330307 9474 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: No bootstrap required, opened a new log
I20260929 02:18:27.330536 9474 ts_tablet_manager.cc:1397] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260929 02:18:27.331374 9475 log.cc:824] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:27.334337 9475 tablet_bootstrap.cc:492] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: No bootstrap required, opened a new log
I20260929 02:18:27.334590 9475 ts_tablet_manager.cc:1397] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:18:27.338343 9474 raft_consensus.cc:348] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.338800 9474 raft_consensus.cc:374] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.338891 9474 raft_consensus.cc:729] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127279b02d4b451c8bef7ef7bf3e55b0, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.339640 9474 consensus_queue.cc:260] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.340266 9476 raft_consensus.cc:348] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.340651 9316 heartbeater.cc:503] Master 127.7.170.62:39439 was elected leader, sending a full tablet report...
I20260929 02:18:27.340667 9476 raft_consensus.cc:374] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.340746 9476 raft_consensus.cc:729] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b0dde18e604bf5b70f6d9067bf77c7, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.341372 9476 consensus_queue.cc:260] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.342576 9474 ts_tablet_manager.cc:1428] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260929 02:18:27.343470 9455 heartbeater.cc:503] Master 127.7.170.62:39439 was elected leader, sending a full tablet report...
I20260929 02:18:27.344696 9475 raft_consensus.cc:348] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.345121 9475 raft_consensus.cc:374] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.345223 9475 raft_consensus.cc:729] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ff696dccef4b59abe921d1f721bf40, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.345255 9476 ts_tablet_manager.cc:1428] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260929 02:18:27.345880 9475 consensus_queue.cc:260] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.347467 9475 ts_tablet_manager.cc:1428] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260929 02:18:27.470371 9456 tablet.cc:2369] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:18:27.490608 9317 tablet.cc:2369] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:27.496183 9480 raft_consensus.cc:482] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:27.496483 9480 raft_consensus.cc:504] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.498136 9480 leader_election.cc:288] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931), b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481)
I20260929 02:18:27.504911 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.498517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:36265 (local address 127.7.170.3:42931)
0929 02:18:27.498709 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.498715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.499477 (+ 762us) server_negotiation.cc:407] Connection header received
0929 02:18:27.499622 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.499626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.499686 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.499760 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.500403 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.501208 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.503004 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.503809 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.503921 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.504009 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.504515 (+ 506us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.504649 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:18:27.504920 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.498711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56351 (local address 127.7.170.1:45481)
0929 02:18:27.498897 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.498903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.499359 (+ 456us) server_negotiation.cc:407] Connection header received
0929 02:18:27.499591 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.499596 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.499678 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.499748 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.500449 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.501241 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.503142 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.503922 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.504056 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.504159 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.504575 (+ 416us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.504701 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:18:27.505052 9483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.498406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45481 (local address 127.7.170.2:56351)
0929 02:18:27.499243 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.499261 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.499429 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.499779 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.499783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.499807 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.500311 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.500320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.501365 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.501369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.502989 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.503002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.504231 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.504236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.504388 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.504426 (+ 38us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.504533 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.504654 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"mutex_wait_us":33,"thread_start_us":469,"threads_started":1}
I20260929 02:18:27.505389 9484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.498529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42931 (local address 127.7.170.2:36265)
0929 02:18:27.499384 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.499398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.499507 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.499784 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.499788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.499840 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.500247 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.500260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.501334 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.501339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.502859 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.502870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.504173 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.504181 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.504334 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.504387 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.504458 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.504573 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":10,"spinlock_wait_cycles":27904,"thread_start_us":524,"threads_started":1}
I20260929 02:18:27.506300 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4c6a74e7d74832870b918816fb6840" candidate_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" is_pre_election: true
I20260929 02:18:27.506500 9132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4c6a74e7d74832870b918816fb6840" candidate_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ff696dccef4b59abe921d1f721bf40" is_pre_election: true
I20260929 02:18:27.506804 9406 raft_consensus.cc:2454] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127279b02d4b451c8bef7ef7bf3e55b0 in term 0.
I20260929 02:18:27.506945 9132 raft_consensus.cc:2454] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 127279b02d4b451c8bef7ef7bf3e55b0 in term 0.
I20260929 02:18:27.507483 9197 leader_election.cc:302] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 127279b02d4b451c8bef7ef7bf3e55b0, 91b0dde18e604bf5b70f6d9067bf77c7, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.507957 9480 raft_consensus.cc:2781] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260929 02:18:27.508054 9178 tablet.cc:2369] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:27.508100 9480 raft_consensus.cc:482] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:27.508214 9480 raft_consensus.cc:3037] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.510421 9480 raft_consensus.cc:504] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.510991 9480 leader_election.cc:288] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [CANDIDATE]: Term 1 election: Requested vote from peers 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931), b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481)
I20260929 02:18:27.511386 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4c6a74e7d74832870b918816fb6840" candidate_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b0dde18e604bf5b70f6d9067bf77c7"
I20260929 02:18:27.511401 9132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b4c6a74e7d74832870b918816fb6840" candidate_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ff696dccef4b59abe921d1f721bf40"
I20260929 02:18:27.511552 9406 raft_consensus.cc:3037] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.511554 9132 raft_consensus.cc:3037] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.513742 9406 raft_consensus.cc:2454] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127279b02d4b451c8bef7ef7bf3e55b0 in term 1.
I20260929 02:18:27.513775 9132 raft_consensus.cc:2454] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 127279b02d4b451c8bef7ef7bf3e55b0 in term 1.
I20260929 02:18:27.514210 9198 leader_election.cc:302] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 127279b02d4b451c8bef7ef7bf3e55b0, 91b0dde18e604bf5b70f6d9067bf77c7, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.514539 9480 raft_consensus.cc:2781] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:27.514951 9480 raft_consensus.cc:686] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 LEADER]: Becoming Leader. State: Replica: 127279b02d4b451c8bef7ef7bf3e55b0, State: Running, Role: LEADER
I20260929 02:18:27.515441 9480 consensus_queue.cc:237] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } }
I20260929 02:18:27.519776 8970 catalog_manager.cc:5495] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:27.541404 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.536275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54390 (local address 127.7.170.2:37999)
0929 02:18:27.536479 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.536486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.536563 (+ 77us) server_negotiation.cc:407] Connection header received
0929 02:18:27.536801 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.536805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.536881 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.537035 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:27.537380 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.538121 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.539421 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.539840 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.539930 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.540347 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:27.540847 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:27.540987 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.541084 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.541148 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:18:27.549234 9406 raft_consensus.cc:1261] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Refusing update from remote peer 127279b02d4b451c8bef7ef7bf3e55b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:27.549234 9132 raft_consensus.cc:1261] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Refusing update from remote peer 127279b02d4b451c8bef7ef7bf3e55b0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:27.550256 9485 consensus_queue.cc:1035] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:27.550571 9480 consensus_queue.cc:1035] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:27.561002 9488 mvcc.cc:205] Tried to move back new op lower bound from 7334495467711528960 to 7334495467586834432. Current Snapshot: MvccSnapshot[applied={T|T < 7334495467711528960}]
I20260929 02:18:27.562057 9489 mvcc.cc:205] Tried to move back new op lower bound from 7334495467711528960 to 7334495467586834432. Current Snapshot: MvccSnapshot[applied={T|T < 7334495467711528960}]
I20260929 02:18:27.579748 8969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:27.580183 8969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:27.584488 8969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Sending DeleteTablet for 3 replicas of tablet 7b4c6a74e7d74832870b918816fb6840
I20260929 02:18:27.585853 9107 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4c6a74e7d74832870b918816fb6840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:27 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:53622
I20260929 02:18:27.585893 9246 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4c6a74e7d74832870b918816fb6840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:27 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:54378
I20260929 02:18:27.585877 9385 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b4c6a74e7d74832870b918816fb6840 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:27 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:39448
I20260929 02:18:27.589983 8969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:27.590791 8969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:27.603116 9106 tablet_service.cc:1467] Processing CreateTablet for tablet ba3135bc1f8f499da021a7e74fc472ad (DEFAULT_TABLE table=test-table [id=3a2e549a18c849d98e152582655591f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.603834 9106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3135bc1f8f499da021a7e74fc472ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.604303 9245 tablet_service.cc:1467] Processing CreateTablet for tablet ba3135bc1f8f499da021a7e74fc472ad (DEFAULT_TABLE table=test-table [id=3a2e549a18c849d98e152582655591f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.605566 9245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3135bc1f8f499da021a7e74fc472ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.611929 9475 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Bootstrap starting.
I20260929 02:18:27.614936 9475 tablet_bootstrap.cc:654] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.618089 9474 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Bootstrap starting.
I20260929 02:18:27.618846 9384 tablet_service.cc:1467] Processing CreateTablet for tablet ba3135bc1f8f499da021a7e74fc472ad (DEFAULT_TABLE table=test-table [id=3a2e549a18c849d98e152582655591f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:27.619519 9384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ba3135bc1f8f499da021a7e74fc472ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.619735 9498 tablet_replica.cc:330] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: stopping tablet replica
I20260929 02:18:27.620462 9474 tablet_bootstrap.cc:654] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.621153 9500 tablet_replica.cc:330] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: stopping tablet replica
I20260929 02:18:27.621537 9499 tablet_replica.cc:330] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: stopping tablet replica
I20260929 02:18:27.623200 9475 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: No bootstrap required, opened a new log
I20260929 02:18:27.623458 9475 ts_tablet_manager.cc:1397] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260929 02:18:27.623518 9499 raft_consensus.cc:2229] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:27.624547 9500 raft_consensus.cc:2229] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:27.626091 9475 raft_consensus.cc:348] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.626389 9475 raft_consensus.cc:374] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.626515 9475 raft_consensus.cc:729] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ff696dccef4b59abe921d1f721bf40, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.626811 9475 consensus_queue.cc:260] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.627609 9498 raft_consensus.cc:2229] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:27.628407 9475 ts_tablet_manager.cc:1428] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Time spent starting tablet: real 0.005s user 0.002s sys 0.001s
I20260929 02:18:27.628455 9498 raft_consensus.cc:2258] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:27.627699 9476 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Bootstrap starting.
I20260929 02:18:27.629240 9500 raft_consensus.cc:2258] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:27.624895 9499 raft_consensus.cc:2258] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:27.632642 9498 ts_tablet_manager.cc:1905] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:27.633635 9476 tablet_bootstrap.cc:654] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.633919 9474 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: No bootstrap required, opened a new log
I20260929 02:18:27.634094 9474 ts_tablet_manager.cc:1397] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20260929 02:18:27.635815 9499 ts_tablet_manager.cc:1905] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:27.635844 9474 raft_consensus.cc:348] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.636059 9474 raft_consensus.cc:374] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.636077 9476 tablet_bootstrap.cc:492] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: No bootstrap required, opened a new log
I20260929 02:18:27.636178 9474 raft_consensus.cc:729] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127279b02d4b451c8bef7ef7bf3e55b0, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.636224 9476 ts_tablet_manager.cc:1397] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260929 02:18:27.636399 9474 consensus_queue.cc:260] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.636583 9500 ts_tablet_manager.cc:1905] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:27.637048 9474 ts_tablet_manager.cc:1428] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:27.637117 9476 raft_consensus.cc:348] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.638419 9476 raft_consensus.cc:374] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.638540 9476 raft_consensus.cc:729] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b0dde18e604bf5b70f6d9067bf77c7, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.638798 9476 consensus_queue.cc:260] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.639447 9476 ts_tablet_manager.cc:1428] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 02:18:27.646044 9498 ts_tablet_manager.cc:1918] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:27.646100 9500 ts_tablet_manager.cc:1918] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:27.646376 9498 log.cc:1196] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/7b4c6a74e7d74832870b918816fb6840
I20260929 02:18:27.646588 9500 log.cc:1196] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/7b4c6a74e7d74832870b918816fb6840
I20260929 02:18:27.647265 9498 ts_tablet_manager.cc:1939] T 7b4c6a74e7d74832870b918816fb6840 P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting consensus metadata
I20260929 02:18:27.647486 9500 ts_tablet_manager.cc:1939] T 7b4c6a74e7d74832870b918816fb6840 P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting consensus metadata
I20260929 02:18:27.647919 9499 ts_tablet_manager.cc:1918] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:27.648157 9499 log.cc:1196] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/7b4c6a74e7d74832870b918816fb6840
I20260929 02:18:27.648975 9499 ts_tablet_manager.cc:1939] T 7b4c6a74e7d74832870b918816fb6840 P b7ff696dccef4b59abe921d1f721bf40: Deleting consensus metadata
I20260929 02:18:27.649796 8957 catalog_manager.cc:4841] TS 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999): tablet 7b4c6a74e7d74832870b918816fb6840 (table test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]) successfully deleted
I20260929 02:18:27.650353 8954 catalog_manager.cc:4841] TS 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931): tablet 7b4c6a74e7d74832870b918816fb6840 (table test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]) successfully deleted
I20260929 02:18:27.651083 8955 catalog_manager.cc:4841] TS b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481): tablet 7b4c6a74e7d74832870b918816fb6840 (table test-table [id=4219ff84e98c4a3c9f4f68f29ceb3b12]) successfully deleted
I20260929 02:18:27.692451 9482 raft_consensus.cc:482] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:27.692730 9482 raft_consensus.cc:504] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.695227 9482 leader_election.cc:288] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999), 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931)
I20260929 02:18:27.706539 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.695639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:46129 (local address 127.7.170.2:37999)
0929 02:18:27.695840 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.695846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.698449 (+ 2603us) server_negotiation.cc:407] Connection header received
0929 02:18:27.698594 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.698600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.698679 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.698783 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.700040 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.701190 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.704508 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.705575 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.705711 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.705823 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.706098 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.706223 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 02:18:27.707324 9505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.696689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37999 (local address 127.7.170.1:46129)
0929 02:18:27.698204 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.698223 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.698459 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.698847 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.698852 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.698877 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.699877 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.699888 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.701341 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.701346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.704307 (+ 2961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.704322 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.705780 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.705798 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.705948 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.705979 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.706060 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.706192 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1325,"mutex_wait_us":84,"spinlock_wait_cycles":1920,"thread_start_us":1055,"threads_started":1}
I20260929 02:18:27.707440 9504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.695845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42931 (local address 127.7.170.1:43573)
0929 02:18:27.697373 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.697397 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.697739 (+ 342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.698404 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.698408 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.698434 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.699313 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.699328 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.701292 (+ 1964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.701298 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.704310 (+ 3012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.704322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.706670 (+ 2348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.706679 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.706851 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.706874 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.706945 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.707035 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1366,"mutex_wait_us":90,"thread_start_us":1025,"threads_started":1}
I20260929 02:18:27.708287 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.695687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:43573 (local address 127.7.170.3:42931)
0929 02:18:27.695886 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.695892 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.697616 (+ 1724us) server_negotiation.cc:407] Connection header received
0929 02:18:27.697935 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.697941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.698071 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.698146 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.699591 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.701126 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.704508 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.706080 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.706201 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.706293 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.706996 (+ 703us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.707129 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:18:27.708766 9271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3135bc1f8f499da021a7e74fc472ad" candidate_uuid: "b7ff696dccef4b59abe921d1f721bf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" is_pre_election: true
I20260929 02:18:27.709051 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3135bc1f8f499da021a7e74fc472ad" candidate_uuid: "b7ff696dccef4b59abe921d1f721bf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" is_pre_election: true
I20260929 02:18:27.709389 9406 raft_consensus.cc:2454] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ff696dccef4b59abe921d1f721bf40 in term 0.
I20260929 02:18:27.709395 9271 raft_consensus.cc:2454] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b7ff696dccef4b59abe921d1f721bf40 in term 0.
I20260929 02:18:27.710251 9060 leader_election.cc:302] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 127279b02d4b451c8bef7ef7bf3e55b0, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.710718 9482 raft_consensus.cc:2781] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:27.710981 9482 raft_consensus.cc:482] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:27.711174 9482 raft_consensus.cc:3037] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.715027 9482 raft_consensus.cc:504] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.715914 9482 leader_election.cc:288] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [CANDIDATE]: Term 1 election: Requested vote from peers 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999), 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931)
I20260929 02:18:27.716318 9271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3135bc1f8f499da021a7e74fc472ad" candidate_uuid: "b7ff696dccef4b59abe921d1f721bf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127279b02d4b451c8bef7ef7bf3e55b0"
I20260929 02:18:27.716444 9406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ba3135bc1f8f499da021a7e74fc472ad" candidate_uuid: "b7ff696dccef4b59abe921d1f721bf40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91b0dde18e604bf5b70f6d9067bf77c7"
I20260929 02:18:27.716601 9271 raft_consensus.cc:3037] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.716622 9406 raft_consensus.cc:3037] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.720227 9406 raft_consensus.cc:2454] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ff696dccef4b59abe921d1f721bf40 in term 1.
I20260929 02:18:27.720470 9271 raft_consensus.cc:2454] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b7ff696dccef4b59abe921d1f721bf40 in term 1.
I20260929 02:18:27.720770 9057 leader_election.cc:302] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b0dde18e604bf5b70f6d9067bf77c7, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.721213 9482 raft_consensus.cc:2781] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:27.722316 9482 raft_consensus.cc:686] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 1 LEADER]: Becoming Leader. State: Replica: b7ff696dccef4b59abe921d1f721bf40, State: Running, Role: LEADER
I20260929 02:18:27.722802 9482 consensus_queue.cc:237] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.727183 8970 catalog_manager.cc:5495] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 reported cstate change: term changed from 0 to 1, leader changed from <none> to b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1). New cstate: current_term: 1 leader_uuid: "b7ff696dccef4b59abe921d1f721bf40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } health_report { overall_health: UNKNOWN } } }
Sep 29 02:18:27 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:27.801084 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.779175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.7.170.62:39439)
0929 02:18:27.785326 (+ 6151us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.785330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.785350 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:18:27.785399 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.785403 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.785470 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.785697 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:27.787278 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.788088 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.789933 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.790516 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.790607 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.794537 (+ 3930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:27.794542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:27.794546 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:27.794574 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:27.797600 (+ 3026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.798530 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.798534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.798536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.798708 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:27.799365 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:27.799368 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:27.799370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:27.799708 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:27.799809 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.800432 (+ 623us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.800632 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6050,"thread_start_us":5938,"threads_started":1}
I20260929 02:18:27.801501 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.779138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39439 (local address 127.0.0.1:59542)
0929 02:18:27.779927 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.779948 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.780237 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.785827 (+ 5590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.785837 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.786778 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:27.787111 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.787125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.788207 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.788212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.789792 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.789807 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.790696 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.790706 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.790897 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.791550 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:27.791573 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:27.794332 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:27.797717 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.797730 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.797738 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.798424 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.798808 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:27.798821 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:27.798825 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:27.799261 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:27.799839 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:27.799848 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:27.800120 (+ 272us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.800349 (+ 229us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.800800 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"mutex_wait_us":124,"thread_start_us":448,"threads_started":1}
I20260929 02:18:27.807487 8970 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:59542:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:18:27.808916 8970 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:27.816054 9246 tablet_service.cc:1467] Processing CreateTablet for tablet b7122f3407654d04a0de3fc0e6742bcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2772022720246dda69896f0cf950a1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:27.816083 9106 tablet_service.cc:1467] Processing CreateTablet for tablet b7122f3407654d04a0de3fc0e6742bcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2772022720246dda69896f0cf950a1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:27.816160 9384 tablet_service.cc:1467] Processing CreateTablet for tablet b7122f3407654d04a0de3fc0e6742bcd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b2772022720246dda69896f0cf950a1d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:27.816622 9384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7122f3407654d04a0de3fc0e6742bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.816697 9106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7122f3407654d04a0de3fc0e6742bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.816737 9246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7122f3407654d04a0de3fc0e6742bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:27.823894 9476 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: Bootstrap starting.
I20260929 02:18:27.826236 9474 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: Bootstrap starting.
I20260929 02:18:27.826961 9476 tablet_bootstrap.cc:654] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.828416 9474 tablet_bootstrap.cc:654] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.831718 9475 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: Bootstrap starting.
I20260929 02:18:27.833637 9475 tablet_bootstrap.cc:654] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:27.838482 9475 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: No bootstrap required, opened a new log
I20260929 02:18:27.838788 9475 ts_tablet_manager.cc:1397] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260929 02:18:27.840106 9474 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: No bootstrap required, opened a new log
I20260929 02:18:27.839967 9475 raft_consensus.cc:348] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.840370 9474 ts_tablet_manager.cc:1397] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260929 02:18:27.840467 9475 raft_consensus.cc:374] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.840662 9475 raft_consensus.cc:729] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ff696dccef4b59abe921d1f721bf40, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.840962 9476 tablet_bootstrap.cc:492] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: No bootstrap required, opened a new log
I20260929 02:18:27.841248 9476 ts_tablet_manager.cc:1397] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent bootstrapping tablet: real 0.017s user 0.011s sys 0.004s
I20260929 02:18:27.841168 9475 consensus_queue.cc:260] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.841671 9474 raft_consensus.cc:348] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.841993 9474 raft_consensus.cc:374] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.841878 9482 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.842131 9474 raft_consensus.cc:729] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 127279b02d4b451c8bef7ef7bf3e55b0, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.842280 9482 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.842345 9476 raft_consensus.cc:348] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.842669 9476 raft_consensus.cc:374] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:27.842561 9474 consensus_queue.cc:260] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.842784 9476 raft_consensus.cc:729] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91b0dde18e604bf5b70f6d9067bf77c7, State: Initialized, Role: FOLLOWER
I20260929 02:18:27.843075 9476 consensus_queue.cc:260] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.843643 9481 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.844135 9481 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.843134 9493 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.845197 9476 ts_tablet_manager.cc:1428] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260929 02:18:27.845287 9493 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.845842 9475 ts_tablet_manager.cc:1428] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260929 02:18:27.846089 9474 ts_tablet_manager.cc:1428] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
I20260929 02:18:27.869094 9481 raft_consensus.cc:482] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:27.869367 9481 raft_consensus.cc:504] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.871197 9481 leader_election.cc:288] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481), 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999)
I20260929 02:18:27.881934 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.872187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:51137 (local address 127.7.170.2:37999)
0929 02:18:27.872363 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.872369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.873112 (+ 743us) server_negotiation.cc:407] Connection header received
0929 02:18:27.873320 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.873325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.873445 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.873509 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.874775 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.875943 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.878834 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.880686 (+ 1852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.880833 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.881028 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.881380 (+ 352us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.881547 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:18:27.882270 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.871566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:49433 (local address 127.7.170.1:45481)
0929 02:18:27.871760 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.871771 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.872557 (+ 786us) server_negotiation.cc:407] Connection header received
0929 02:18:27.872854 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.872862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.873039 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.873131 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:27.874064 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.875594 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.879191 (+ 3597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.881089 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.881195 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.881349 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.881720 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.881846 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:18:27.882477 9518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.871256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45481 (local address 127.7.170.3:49433)
0929 02:18:27.872288 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.872311 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.872680 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.873196 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.873200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.873222 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.873806 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.873817 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.875757 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.875763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.879010 (+ 3247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.879027 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.881409 (+ 2382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.881415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.881579 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.881602 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.881680 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.881821 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":864,"mutex_wait_us":29,"thread_start_us":613,"threads_started":1}
I20260929 02:18:27.883059 9519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.871539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37999 (local address 127.7.170.3:51137)
0929 02:18:27.872893 (+ 1354us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.872907 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.873160 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.873787 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.873790 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.873825 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:27.874617 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.874626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.876099 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.876104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.878662 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.878674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.880881 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.880894 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.881178 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.881214 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.882228 (+ 1014us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.882370 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1226,"mutex_wait_us":1,"thread_start_us":965,"threads_started":1}
I20260929 02:18:27.883450 9132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7122f3407654d04a0de3fc0e6742bcd" candidate_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ff696dccef4b59abe921d1f721bf40" is_pre_election: true
I20260929 02:18:27.883749 9132 raft_consensus.cc:2454] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b0dde18e604bf5b70f6d9067bf77c7 in term 0.
I20260929 02:18:27.883947 9271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7122f3407654d04a0de3fc0e6742bcd" candidate_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" is_pre_election: true
I20260929 02:18:27.884323 9271 raft_consensus.cc:2454] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91b0dde18e604bf5b70f6d9067bf77c7 in term 0.
I20260929 02:18:27.884338 9337 leader_election.cc:302] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91b0dde18e604bf5b70f6d9067bf77c7, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.885002 9481 raft_consensus.cc:2781] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:27.885213 9481 raft_consensus.cc:482] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:27.885408 9481 raft_consensus.cc:3037] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.889793 9481 raft_consensus.cc:504] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.891125 9481 leader_election.cc:288] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [CANDIDATE]: Term 1 election: Requested vote from peers b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481), 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999)
I20260929 02:18:27.891426 9271 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7122f3407654d04a0de3fc0e6742bcd" candidate_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "127279b02d4b451c8bef7ef7bf3e55b0"
I20260929 02:18:27.891521 9132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7122f3407654d04a0de3fc0e6742bcd" candidate_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ff696dccef4b59abe921d1f721bf40"
I20260929 02:18:27.891763 9132 raft_consensus.cc:3037] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.891801 9271 raft_consensus.cc:3037] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:27.895641 9132 raft_consensus.cc:2454] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b0dde18e604bf5b70f6d9067bf77c7 in term 1.
I20260929 02:18:27.895812 9271 raft_consensus.cc:2454] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91b0dde18e604bf5b70f6d9067bf77c7 in term 1.
I20260929 02:18:27.896260 9337 leader_election.cc:302] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 127279b02d4b451c8bef7ef7bf3e55b0, 91b0dde18e604bf5b70f6d9067bf77c7, b7ff696dccef4b59abe921d1f721bf40; no voters:
I20260929 02:18:27.896783 9481 raft_consensus.cc:2781] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:27.897751 9481 raft_consensus.cc:686] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 LEADER]: Becoming Leader. State: Replica: 91b0dde18e604bf5b70f6d9067bf77c7, State: Running, Role: LEADER
I20260929 02:18:27.898346 9481 consensus_queue.cc:237] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } }
I20260929 02:18:27.901579 9520 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: TxnStatusTablet state changed. Reason: New leader 91b0dde18e604bf5b70f6d9067bf77c7. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.902305 9520 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:27.903630 8968 catalog_manager.cc:5495] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3). New cstate: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } health_report { overall_health: HEALTHY } } }
I20260929 02:18:27.903864 9522 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:18:27.904156 9522 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:18:27.936144 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.925943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.7.170.3:42931)
0929 02:18:27.926146 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.926152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.926775 (+ 623us) server_negotiation.cc:407] Connection header received
0929 02:18:27.927092 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.927098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.927249 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.927431 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:27.928177 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.929868 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.932303 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.933425 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.933574 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.934353 (+ 779us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:27.935325 (+ 972us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:27.935493 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.935614 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.935702 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:18:27.936216 9514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.926215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42931 (local address 127.0.0.1:39456)
0929 02:18:27.926669 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:27.926690 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:27.926863 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:27.927625 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:27.927630 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:27.927643 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:27.927999 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.928014 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.930010 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.930015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.932136 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:27.932152 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.933662 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.933670 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:27.933996 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.934113 (+ 117us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:27.935457 (+ 1344us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:27.935481 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:18:27.935577 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 02:18:27.935774 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237}
I20260929 02:18:27.942272 9132 raft_consensus.cc:1261] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Refusing update from remote peer 91b0dde18e604bf5b70f6d9067bf77c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:27.942252 9271 raft_consensus.cc:1261] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Refusing update from remote peer 91b0dde18e604bf5b70f6d9067bf77c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:27.943117 9481 consensus_queue.cc:1035] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:27.943996 9520 consensus_queue.cc:1035] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:27.956786 9482 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: TxnStatusTablet state changed. Reason: New leader 91b0dde18e604bf5b70f6d9067bf77c7. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.959683 9482 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.960985 9520 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.961329 9520 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:27.963125 9493 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: TxnStatusTablet state changed. Reason: New leader 91b0dde18e604bf5b70f6d9067bf77c7. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.963557 9493 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.964984 9493 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.965233 9493 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 127279b02d4b451c8bef7ef7bf3e55b0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.966009 9490 mvcc.cc:205] Tried to move back new op lower bound from 7334495469320339456 to 7334495469157117952. Current Snapshot: MvccSnapshot[applied={T|T < 7334495469320339456 or (T in {7334495469320339456})}]
I20260929 02:18:27.966753 9481 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.967144 9481 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P 91b0dde18e604bf5b70f6d9067bf77c7: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:27.969533 9482 tablet_replica.cc:439] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ff696dccef4b59abe921d1f721bf40" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45481 } } peers { permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 } } peers { permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 } } }
I20260929 02:18:27.970139 9482 tablet_replica.cc:442] T b7122f3407654d04a0de3fc0e6742bcd P b7ff696dccef4b59abe921d1f721bf40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:27.982893 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.974262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.7.170.1:45481)
0929 02:18:27.974417 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.974422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:27.974454 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:18:27.974544 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:27.974549 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:27.974687 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:27.974859 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:27.975375 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.976745 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.979395 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:27.980415 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:27.980617 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:27.981384 (+ 767us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:27.982163 (+ 779us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:27.982317 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:27.982423 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 02:18:27.982508 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 02:18:28.012035 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.998725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:58515 (local address 127.7.170.3:42931)
0929 02:18:27.998959 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:27.998965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.000580 (+ 1615us) server_negotiation.cc:407] Connection header received
0929 02:18:28.000825 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.000831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.000956 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.001059 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:28.001948 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.003671 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.006682 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.008983 (+ 2301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.009171 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.011033 (+ 1862us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:28.011188 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:28.011282 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.011488 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.011591 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:18:28.012306 9529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:27.998972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42931 (local address 127.7.170.1:58515)
0929 02:18:28.000323 (+ 1351us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:28.000344 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:28.000657 (+ 313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:28.001085 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:28.001090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:28.001103 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:28.001625 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:28.001643 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.003818 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.003823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:28.006517 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:28.006534 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.010146 (+ 3612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.010155 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:28.010690 (+ 535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.010776 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:28.011304 (+ 528us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:28.011349 (+ 45us) client_negotiation.cc:769] Sending connection context
0929 02:18:28.011439 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:18:28.011610 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1110,"thread_start_us":844,"threads_started":1}
I20260929 02:18:28.026785 9406 raft_consensus.cc:1261] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Refusing update from remote peer b7ff696dccef4b59abe921d1f721bf40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:28.026886 9270 raft_consensus.cc:1261] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Refusing update from remote peer b7ff696dccef4b59abe921d1f721bf40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:28.027907 9506 consensus_queue.cc:1035] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "127279b02d4b451c8bef7ef7bf3e55b0" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:28.028692 9482 consensus_queue.cc:1035] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91b0dde18e604bf5b70f6d9067bf77c7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:28.088369 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.076239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41885 (local address 127.7.170.1:45481)
0929 02:18:28.076456 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.076462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.078197 (+ 1735us) server_negotiation.cc:407] Connection header received
0929 02:18:28.078542 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.078548 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.078637 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.078704 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:28.080094 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.081444 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.084591 (+ 3147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.085698 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.085835 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.087230 (+ 1395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:28.087420 (+ 190us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:28.087516 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.087726 (+ 210us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.087881 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:18:28.089461 9533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.076693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45481 (local address 127.7.170.3:41885)
0929 02:18:28.077964 (+ 1271us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:28.077985 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:28.078350 (+ 365us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:28.078908 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:28.078912 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:28.078925 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:28.079892 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:28.079907 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.081585 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.081591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:28.084430 (+ 2839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:28.084445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.086011 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.086020 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:28.086288 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.086366 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:28.087569 (+ 1203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:28.087582 (+ 13us) client_negotiation.cc:769] Sending connection context
0929 02:18:28.088565 (+ 983us) client_negotiation.cc:240] Negotiation successful
0929 02:18:28.088711 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":968,"thread_start_us":687,"threads_started":1}
I20260929 02:18:28.149120 8970 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:28.149379 8970 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:28.152994 8970 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a84924cf52414b4e921fa77b0eee425c: Sending DeleteTablet for 3 replicas of tablet ba3135bc1f8f499da021a7e74fc472ad
I20260929 02:18:28.154321 9106 tablet_service.cc:1514] Processing DeleteTablet for tablet ba3135bc1f8f499da021a7e74fc472ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:28 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:53622
I20260929 02:18:28.154757 9384 tablet_service.cc:1514] Processing DeleteTablet for tablet ba3135bc1f8f499da021a7e74fc472ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:28 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:39448
I20260929 02:18:28.156173 9246 tablet_service.cc:1514] Processing DeleteTablet for tablet ba3135bc1f8f499da021a7e74fc472ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:28 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:54378
I20260929 02:18:28.166106 9540 tablet_replica.cc:330] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: stopping tablet replica
I20260929 02:18:28.166383 9540 raft_consensus.cc:2229] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:28.166766 9540 raft_consensus.cc:2258] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:28.169749 9540 ts_tablet_manager.cc:1905] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:28.182072 9539 tablet_replica.cc:330] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: stopping tablet replica
I20260929 02:18:28.182382 9539 raft_consensus.cc:2229] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:28.182833 9539 raft_consensus.cc:2258] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:28.184209 9539 ts_tablet_manager.cc:1905] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:28.184576 9538 tablet_replica.cc:330] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: stopping tablet replica
I20260929 02:18:28.184875 9538 raft_consensus.cc:2229] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:28.185606 9538 raft_consensus.cc:2258] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:28.188207 9538 ts_tablet_manager.cc:1905] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:28.194531 9539 ts_tablet_manager.cc:1918] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:28.194763 9539 log.cc:1196] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/ba3135bc1f8f499da021a7e74fc472ad
I20260929 02:18:28.195457 9539 ts_tablet_manager.cc:1939] T ba3135bc1f8f499da021a7e74fc472ad P 91b0dde18e604bf5b70f6d9067bf77c7: Deleting consensus metadata
I20260929 02:18:28.195762 9540 ts_tablet_manager.cc:1918] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:28.195959 9540 log.cc:1196] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/ba3135bc1f8f499da021a7e74fc472ad
I20260929 02:18:28.196651 9540 ts_tablet_manager.cc:1939] T ba3135bc1f8f499da021a7e74fc472ad P 127279b02d4b451c8bef7ef7bf3e55b0: Deleting consensus metadata
I20260929 02:18:28.197177 8954 catalog_manager.cc:4841] TS 91b0dde18e604bf5b70f6d9067bf77c7 (127.7.170.3:42931): tablet ba3135bc1f8f499da021a7e74fc472ad (table test-table [id=3a2e549a18c849d98e152582655591f2]) successfully deleted
I20260929 02:18:28.205130 9538 ts_tablet_manager.cc:1918] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:28.205358 9538 log.cc:1196] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/ba3135bc1f8f499da021a7e74fc472ad
I20260929 02:18:28.206135 9538 ts_tablet_manager.cc:1939] T ba3135bc1f8f499da021a7e74fc472ad P b7ff696dccef4b59abe921d1f721bf40: Deleting consensus metadata
I20260929 02:18:28.207399 8955 catalog_manager.cc:4841] TS b7ff696dccef4b59abe921d1f721bf40 (127.7.170.1:45481): tablet ba3135bc1f8f499da021a7e74fc472ad (table test-table [id=3a2e549a18c849d98e152582655591f2]) successfully deleted
I20260929 02:18:28.209872 8957 catalog_manager.cc:4841] TS 127279b02d4b451c8bef7ef7bf3e55b0 (127.7.170.2:37999): tablet ba3135bc1f8f499da021a7e74fc472ad (table test-table [id=3a2e549a18c849d98e152582655591f2]) successfully deleted
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:28.225519 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.167327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53638 (local address 127.7.170.1:45481)
0929 02:18:28.167514 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.167520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.179678 (+ 12158us) server_negotiation.cc:407] Connection header received
0929 02:18:28.181264 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.181271 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.181387 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.181609 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:28.197099 (+ 15490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.198236 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.200506 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.201584 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.201713 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.213628 (+ 11915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:28.213649 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:28.213670 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:28.213725 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:28.219668 (+ 5943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.220914 (+ 1246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.220955 (+ 41us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.220958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.221202 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.221778 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.221784 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.221787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.222372 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:28.222745 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.223436 (+ 691us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.223669 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260929 02:18:28.226398 9172 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53638
I20260929 02:18:28.277232 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.247407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.7.170.62:39439)
0929 02:18:28.247611 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.247617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.257526 (+ 9909us) server_negotiation.cc:407] Connection header received
0929 02:18:28.258010 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.258017 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.258121 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.258197 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:28.261679 (+ 3482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.263242 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.266094 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.267016 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.267124 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.270034 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:28.270041 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:28.270045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:28.270087 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:28.273711 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.274696 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.274702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.274706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.274864 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.275363 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.275368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.275372 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.275852 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:28.276079 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.276656 (+ 577us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.276894 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260929 02:18:28.278508 8970 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59554
I20260929 02:18:28.281663 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9041
I20260929 02:18:28.323889 9473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.282795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.7.170.1:45481)
0929 02:18:28.283343 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.283350 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.302339 (+ 18989us) server_negotiation.cc:407] Connection header received
0929 02:18:28.302803 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.302810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.302887 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.302964 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:28.304688 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.305875 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.307810 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.308556 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.308660 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.314565 (+ 5905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:28.314572 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:28.314576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:28.314615 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:28.318072 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.321292 (+ 3220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.321298 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.321302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.321457 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.321931 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.321936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.321939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.322282 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:28.322541 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.323218 (+ 677us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.323465 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414}
W20260929 02:18:28.325234 9172 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53652
W20260929 02:18:28.325771 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9041: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:28.325971 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9041
I20260929 02:18:28.348578 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9181
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:28.365674 9470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.349379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.7.170.2:37999)
0929 02:18:28.349558 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.349563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.349675 (+ 112us) server_negotiation.cc:407] Connection header received
0929 02:18:28.349882 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.349887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.349973 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.350075 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:28.351491 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.352551 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.354281 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.354936 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.355081 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.360013 (+ 4932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:28.360027 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:28.360044 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:28.360084 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:28.362947 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.363686 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.363692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.363694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.363817 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.364159 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.364161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.364163 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.364470 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:28.364722 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.365175 (+ 453us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.365336 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 02:18:28.366277 9311 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54404
W20260929 02:18:28.366871 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9181: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:28.367053 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9181
I20260929 02:18:28.385118 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9320
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[8925](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:28.401221 9471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:28.385872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.7.170.3:42931)
0929 02:18:28.386032 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:28.386039 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:28.386108 (+ 69us) server_negotiation.cc:407] Connection header received
0929 02:18:28.386286 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:28.386291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:28.386366 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:28.386451 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:28.387599 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.388321 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.389596 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:28.390063 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:28.390178 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:28.394311 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:28.394324 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:28.394342 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:28.394398 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:28.398479 (+ 4081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.399113 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.399123 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.399127 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.399314 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:28.399645 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:28.399654 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:28.399657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:28.400093 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:28.400419 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:28.400567 (+ 148us) server_negotiation.cc:299] Negotiation successful
0929 02:18:28.400773 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:18:28.401948 9450 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39458
W20260929 02:18:28.402545 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9320: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:28.402668 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9320
I20260929 02:18:28.418231 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 8938
I20260929 02:18:28.419219 9010 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:28.530992 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 8938
2026-09-29T02:18:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4430 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[9549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[9549](info): set up 2 sockets
Sep 29 02:18:28 dist-test-slave-8cs2 krb5kdc[9549](info): commencing operation
krb5kdc: starting...
W20260929 02:18:29.521189 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.951s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:29 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:29.689232 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36311
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:36311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:29.988555 9562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:29.988827 9562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:29.988955 9562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:29.995782 9562 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:29.995877 9562 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:18:29.995945 9562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:29.996029 9562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:29.996093 9562 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:29.996171 9562 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:30.005517 9562 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:36311
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:30.006707 9562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:30.007936 9562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:30.019011 9567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:30.018919 9568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:30.018944 9570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:30.019835 9562 server_base.cc:1034] running on GCE node
I20260929 02:18:30.020677 9562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:30.022368 9562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:30.023609 9562 hybrid_clock.cc:648] HybridClock initialized: now 1790648310023537 us; error 45 us; skew 500 ppm
I20260929 02:18:30.023928 9562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:30 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:30.030862 9562 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:30.032316 9562 webserver.cc:460] Webserver started at http://127.7.170.62:35063/ using document root <none> and password file <none>
I20260929 02:18:30.033027 9562 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:30.033123 9562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:30.033416 9562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:30.036178 9562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "701a8f36869b44dbbd4eb2a92f3115b4"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "87a00701ce6d40a333b889c7aa74aad4"
server_key_iv: "b42c785c1a637cc56e697bb5c5bd9105"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.036891 9562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "701a8f36869b44dbbd4eb2a92f3115b4"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "87a00701ce6d40a333b889c7aa74aad4"
server_key_iv: "b42c785c1a637cc56e697bb5c5bd9105"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.041620 9562 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:18:30.044699 9577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:30.045980 9562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:30.046128 9562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "701a8f36869b44dbbd4eb2a92f3115b4"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "87a00701ce6d40a333b889c7aa74aad4"
server_key_iv: "b42c785c1a637cc56e697bb5c5bd9105"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.046327 9562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:30.060887 9562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:30.064808 9562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:30.065055 9562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:30.084726 9562 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:36311
I20260929 02:18:30.084749 9638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:36311 every 8 connection(s)
I20260929 02:18:30.086035 9562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:30.089617 9639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:30.093955 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9562
I20260929 02:18:30.094156 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:30.094916 7848 external_mini_cluster.cc:1428] Setting key ad8a2d2be4476a891992a3ed805e80fe
I20260929 02:18:30.101081 9639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Bootstrap starting.
I20260929 02:18:30.104547 9639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:30.105671 9639 log.cc:824] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:30.108357 9639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: No bootstrap required, opened a new log
Sep 29 02:18:30 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:30.115947 9639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } }
I20260929 02:18:30.116279 9639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:30.116374 9639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701a8f36869b44dbbd4eb2a92f3115b4, State: Initialized, Role: FOLLOWER
I20260929 02:18:30.116763 9642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:30.096899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.7.170.62:36311)
0929 02:18:30.097680 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:30.097692 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:30.097730 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:18:30.099547 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:30.099570 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:30.099962 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:30.100372 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:30.101983 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.103257 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.105042 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.105768 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.105962 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:30.109821 (+ 3859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:30.109834 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:30.109848 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:30.109890 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:30.112782 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.113454 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.113461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.113478 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.113641 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.114001 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.114004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.114006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.114546 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:30.114833 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:30.115257 (+ 424us) server_negotiation.cc:299] Negotiation successful
0929 02:18:30.115693 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":322,"threads_started":1}
I20260929 02:18:30.116962 9639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } }
I20260929 02:18:30.117199 9639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:30.117327 9639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:30.117491 9639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:30.119796 9639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } }
I20260929 02:18:30.120256 9639 leader_election.cc:302] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 701a8f36869b44dbbd4eb2a92f3115b4; no voters:
I20260929 02:18:30.120774 9639 leader_election.cc:288] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:30.120899 9644 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:30.121716 9644 raft_consensus.cc:686] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [term 1 LEADER]: Becoming Leader. State: Replica: 701a8f36869b44dbbd4eb2a92f3115b4, State: Running, Role: LEADER
I20260929 02:18:30.122192 9644 consensus_queue.cc:237] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } }
I20260929 02:18:30.123703 9644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 701a8f36869b44dbbd4eb2a92f3115b4. Latest consensus state: current_term: 1 leader_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } } }
I20260929 02:18:30.124068 9644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:30.124408 9646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701a8f36869b44dbbd4eb2a92f3115b4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36311 } } }
I20260929 02:18:30.124795 9646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:30.128499 9639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:30.131377 9648 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:30.136186 9648 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:30.144908 9648 catalog_manager.cc:1347] Generated new cluster ID: 80adfd0c042c419d8e3d12f21ef510b1
I20260929 02:18:30.145064 9648 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:30.166914 9648 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:30.168889 9648 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:30.176715 9648 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Generated new TSK 0
I20260929 02:18:30.177781 9648 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:30.222394 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36311
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:30.529920 9665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:30.530191 9665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:30.530332 9665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:30.537421 9665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:30.537533 9665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:30.537657 9665 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:18:30.537722 9665 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:30.537822 9665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:30.547166 9665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:30.548328 9665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:30.549659 9665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:30.561474 9673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:30.561625 9670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:30.561501 9671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:30.562378 9665 server_base.cc:1034] running on GCE node
I20260929 02:18:30.562942 9665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:30.563766 9665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:30.565016 9665 hybrid_clock.cc:648] HybridClock initialized: now 1790648310564970 us; error 39 us; skew 500 ppm
I20260929 02:18:30.565294 9665 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:30 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:30.568181 9665 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:30.569540 9665 webserver.cc:460] Webserver started at http://127.7.170.1:40045/ using document root <none> and password file <none>
I20260929 02:18:30.570117 9665 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:30.570209 9665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:30.570448 9665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:30.573094 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "5f6e0096fa304161b1ed7875335d514d"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "a1fe185d36fc0174930a424a0448aed1"
server_key_iv: "919ad78795d5b157cc3dc7bbfbc60159"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.573745 9665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "5f6e0096fa304161b1ed7875335d514d"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "a1fe185d36fc0174930a424a0448aed1"
server_key_iv: "919ad78795d5b157cc3dc7bbfbc60159"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.578341 9665 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:18:30.581790 9680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:30.583006 9665 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:30.583132 9665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "5f6e0096fa304161b1ed7875335d514d"
format_stamp: "Formatted at 2026-09-29 02:18:30 on dist-test-slave-8cs2"
server_key: "a1fe185d36fc0174930a424a0448aed1"
server_key_iv: "919ad78795d5b157cc3dc7bbfbc60159"
server_key_version: "encryptionkey@0"
I20260929 02:18:30.583300 9665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:30.612165 9665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:30.616005 9665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:30.616264 9665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:30.618935 9665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:30.619023 9665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:30.619131 9665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:30.619208 9665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:30 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:30.640151 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:30.619353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:60825 (local address 127.7.170.62:36311)
0929 02:18:30.619859 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:30.619865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:30.620997 (+ 1132us) server_negotiation.cc:407] Connection header received
0929 02:18:30.623005 (+ 2008us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:30.623012 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:30.623092 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:30.623214 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:30.625163 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.626112 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.628352 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.628753 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.628878 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:30.633801 (+ 4923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:30.633807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:30.633812 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:30.633851 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:30.637104 (+ 3253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.637760 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.637766 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.637770 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.637912 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.638301 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.638306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.638309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.638660 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:30.638807 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:30.639430 (+ 623us) server_negotiation.cc:299] Negotiation successful
0929 02:18:30.639662 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260929 02:18:30.640856 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:30.620016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.1:60825)
0929 02:18:30.620813 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:30.620861 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:30.622655 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:30.623447 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:30.623455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:30.624116 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:30.624976 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:30.625002 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.626261 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.626266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:30.628189 (+ 1923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:30.628209 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.629056 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.629068 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:30.629438 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:30.630176 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:30.630213 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:30.633571 (+ 3358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:30.637234 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:30.637241 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:30.637253 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:30.637647 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:30.638017 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:30.638020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:30.638022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:30.638206 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:30.638766 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:30.638771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:30.639094 (+ 323us) client_negotiation.cc:769] Sending connection context
0929 02:18:30.639306 (+ 212us) client_negotiation.cc:240] Negotiation successful
0929 02:18:30.639698 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":319,"threads_started":1}
I20260929 02:18:30.658242 9749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:30.646177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.1:48655)
0929 02:18:30.646836 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:30.646857 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:30.647002 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:30.647322 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:30.647326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:30.647924 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:30.648398 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:30.648417 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.649624 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.649629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:30.650791 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:30.650803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.651494 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.651500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:30.651659 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:30.652233 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:30.652248 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:30.652847 (+ 599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:30.655917 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:30.655921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:30.655925 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:30.656394 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:30.656790 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:30.656793 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:30.656795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:30.656901 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:30.657471 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:30.657474 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:30.657613 (+ 139us) client_negotiation.cc:769] Sending connection context
0929 02:18:30.657730 (+ 117us) client_negotiation.cc:240] Negotiation successful
0929 02:18:30.657936 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":350,"threads_started":1}
I20260929 02:18:30.658425 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:30.646163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:48655 (local address 127.7.170.62:36311)
0929 02:18:30.646340 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:30.646345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:30.646952 (+ 607us) server_negotiation.cc:407] Connection header received
0929 02:18:30.647123 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:30.647128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:30.647204 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:30.647296 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:30.648523 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.649500 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.650914 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:30.651328 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:30.651433 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:30.653027 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:30.653033 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:30.653038 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:30.653067 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:30.655759 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.656522 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.656528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.656532 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.656683 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:30.657030 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:30.657035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:30.657039 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:30.657365 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:30.657523 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:30.657847 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 02:18:30.658071 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:18:30.668376 9665 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46205
I20260929 02:18:30.668416 9800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46205 every 8 connection(s)
I20260929 02:18:30.669621 9665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:30.670495 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9665
I20260929 02:18:30.670683 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:30.671278 7848 external_mini_cluster.cc:1428] Setting key 8bd432771cd62b5eb92068602e6284fb
I20260929 02:18:30.671619 9801 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36311
I20260929 02:18:30.671832 9801 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:30.672437 9801 heartbeater.cc:511] Master 127.7.170.62:36311 requested a full tablet report, sending...
I20260929 02:18:30.674288 9594 ts_manager.cc:194] Registered new tserver with Master: 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205)
I20260929 02:18:30.675704 9594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:60825
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:30.711519 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36311
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:31.014818 9805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:31.015089 9805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:31.015218 9805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:31.022138 9805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:31.022236 9805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:31.022356 9805 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:18:31.022415 9805 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:31.022516 9805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:31.032887 9805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:31.034181 9805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:31.035449 9805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:31.047492 9810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:31.047533 9811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:31.047626 9813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:31.048170 9805 server_base.cc:1034] running on GCE node
I20260929 02:18:31.048707 9805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:31.049726 9805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:31.050913 9805 hybrid_clock.cc:648] HybridClock initialized: now 1790648311050849 us; error 54 us; skew 500 ppm
I20260929 02:18:31.051198 9805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:31.054011 9805 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:31.055265 9805 webserver.cc:460] Webserver started at http://127.7.170.2:43695/ using document root <none> and password file <none>
I20260929 02:18:31.055886 9805 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:31.055996 9805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:31.056273 9805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:31.059221 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "32d9596a1aeb46eabf7e374ccb58a1fa"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "4bdf7c27603571a4c20dbd08764ae9ff"
server_key_iv: "f40043c85e69b46860c0f4e17ebfa7c1"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.059953 9805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "32d9596a1aeb46eabf7e374ccb58a1fa"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "4bdf7c27603571a4c20dbd08764ae9ff"
server_key_iv: "f40043c85e69b46860c0f4e17ebfa7c1"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.064666 9805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260929 02:18:31.067811 9820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:31.069101 9805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:31.069259 9805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "32d9596a1aeb46eabf7e374ccb58a1fa"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "4bdf7c27603571a4c20dbd08764ae9ff"
server_key_iv: "f40043c85e69b46860c0f4e17ebfa7c1"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.069437 9805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:31.086182 9805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:31.090166 9805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:31.090420 9805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:31.093048 9805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:31.093195 9805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:31.093333 9805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:31.093468 9805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:31.116878 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.093635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:59381 (local address 127.7.170.62:36311)
0929 02:18:31.093789 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.093795 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.095249 (+ 1454us) server_negotiation.cc:407] Connection header received
0929 02:18:31.098116 (+ 2867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.098122 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.098240 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.098353 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:31.100754 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.101931 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.104818 (+ 2887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.105398 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.105525 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.110175 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:31.110180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:31.110183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:31.110217 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:31.113218 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.114109 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.114115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.114119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.114300 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.114820 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.114826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.114830 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.115196 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:31.115360 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.116235 (+ 875us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.116485 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:18:31.117687 9829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.094260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.2:59381)
0929 02:18:31.095104 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.095153 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.097803 (+ 2650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.098746 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.098758 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.099527 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:31.100526 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.100555 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.102127 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.102137 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.104641 (+ 2504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.104660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.105767 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.105772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.106029 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.106766 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:31.106799 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:31.109957 (+ 3158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:31.113352 (+ 3395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.113365 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.113382 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.113974 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.114434 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.114439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.114442 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.114701 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.115326 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:31.115334 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:31.115777 (+ 443us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.116140 (+ 363us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.116540 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":359,"threads_started":1}
I20260929 02:18:31.136893 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.122792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:52981 (local address 127.7.170.62:36311)
0929 02:18:31.123008 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.123014 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.123529 (+ 515us) server_negotiation.cc:407] Connection header received
0929 02:18:31.123652 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.123658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.123734 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.123838 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:31.124643 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.125817 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.128042 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.128560 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.128678 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.130799 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:31.130805 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:31.130809 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:31.130853 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:31.134146 (+ 3293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.134904 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.134910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.134913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.135037 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.135451 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.135454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.135456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.135859 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:31.136015 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.136478 (+ 463us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.136624 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:18:31.137247 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.122868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.2:52981)
0929 02:18:31.123447 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.123461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.123558 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.123863 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.123866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.124265 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:31.124527 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.124535 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.126002 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.126011 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.127891 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.127911 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.128872 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.128876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.129039 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.129656 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:31.129678 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:31.130639 (+ 961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:31.134260 (+ 3621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.134268 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.134273 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.134793 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.135137 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.135141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.135148 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.135300 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.136008 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:31.136013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:31.136207 (+ 194us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.136391 (+ 184us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.136709 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":304,"threads_started":1}
I20260929 02:18:31.138270 9805 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:32773
I20260929 02:18:31.138363 9939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:32773 every 8 connection(s)
I20260929 02:18:31.139849 9805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:31.141789 9940 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36311
I20260929 02:18:31.141989 9940 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:31.142616 9940 heartbeater.cc:511] Master 127.7.170.62:36311 requested a full tablet report, sending...
I20260929 02:18:31.143764 9594 ts_manager.cc:194] Registered new tserver with Master: 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773)
I20260929 02:18:31.144507 9594 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:59381
I20260929 02:18:31.147714 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9805
I20260929 02:18:31.147866 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:31.148267 7848 external_mini_cluster.cc:1428] Setting key 61f5560d4a1f5b8ee82797225c60c3d5
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:31.183908 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36311
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:31.481676 9944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:31.481943 9944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:31.482070 9944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:31.489159 9944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:31.489253 9944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:31.489372 9944 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260929 02:18:31.489431 9944 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:18:31.489528 9944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:31.498818 9944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:36311
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:31.500042 9944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:31.501305 9944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:31.513437 9949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:31.513412 9952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:31.513469 9950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:31.514321 9944 server_base.cc:1034] running on GCE node
I20260929 02:18:31.514906 9944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:31.515761 9944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:31.517007 9944 hybrid_clock.cc:648] HybridClock initialized: now 1790648311516954 us; error 36 us; skew 500 ppm
I20260929 02:18:31.517292 9944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:31.520133 9944 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:31.521469 9944 webserver.cc:460] Webserver started at http://127.7.170.3:37497/ using document root <none> and password file <none>
I20260929 02:18:31.522095 9944 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:31.522203 9944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:31.522478 9944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:31.525177 9944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "03a29061cee741b5920488c85553f3c6"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "b499cb1b0a1740014a33791f3ed2b604"
server_key_iv: "c95892a08524e8b7a5054a2edc1dfd71"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.525885 9944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "03a29061cee741b5920488c85553f3c6"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "b499cb1b0a1740014a33791f3ed2b604"
server_key_iv: "c95892a08524e8b7a5054a2edc1dfd71"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.530665 9944 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:18:31.533900 9959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:31.535142 9944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:31.535293 9944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "03a29061cee741b5920488c85553f3c6"
format_stamp: "Formatted at 2026-09-29 02:18:31 on dist-test-slave-8cs2"
server_key: "b499cb1b0a1740014a33791f3ed2b604"
server_key_iv: "c95892a08524e8b7a5054a2edc1dfd71"
server_key_version: "encryptionkey@0"
I20260929 02:18:31.535465 9944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:31.564975 9944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:31.568681 9944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:31.568974 9944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:31.571424 9944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:31.571532 9944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:31.571635 9944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:31.571713 9944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:31.592664 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.572036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35703 (local address 127.7.170.62:36311)
0929 02:18:31.572200 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.572207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.573726 (+ 1519us) server_negotiation.cc:407] Connection header received
0929 02:18:31.575747 (+ 2021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.575753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.575835 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.575951 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:31.577869 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.578846 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.581184 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.581695 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.581813 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.586032 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:31.586039 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:31.586043 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:31.586077 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:31.589125 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.589989 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.589995 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.589998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.590137 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.590678 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.590683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.590687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.590950 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:31.591107 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.592083 (+ 976us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.592312 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:18:31.594228 9968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.572705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.3:35703)
0929 02:18:31.573567 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.573621 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.575453 (+ 1832us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.576169 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.576177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.576831 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:31.577690 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.577719 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.578982 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.578987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.581052 (+ 2065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.581067 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.581942 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.581950 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.582163 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.582855 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:31.582880 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:31.585736 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:31.589267 (+ 3531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.589275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.589290 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.589865 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.590272 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.590277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.590281 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.590561 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.591108 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:31.591116 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:31.591606 (+ 490us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.591956 (+ 350us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.592543 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":346,"threads_started":1}
I20260929 02:18:31.613646 9944 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:35439
I20260929 02:18:31.613685 10078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:35439 every 8 connection(s)
I20260929 02:18:31.614812 9944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:31.616751 10079 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36311
I20260929 02:18:31.616998 10079 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:31.617799 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.601146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:56227 (local address 127.7.170.62:36311)
0929 02:18:31.601315 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.601322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.605957 (+ 4635us) server_negotiation.cc:407] Connection header received
0929 02:18:31.606077 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.606083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.606167 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.606277 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:31.607041 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.608190 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.609704 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.610296 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.610438 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.612629 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:31.612636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:31.612641 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:31.612679 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:31.615176 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.616033 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.616038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.616042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.616208 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.616551 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.616556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.616559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.616863 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:31.617018 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.617425 (+ 407us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.617575 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260929 02:18:31.617769 10079 heartbeater.cc:511] Master 127.7.170.62:36311 requested a full tablet report, sending...
I20260929 02:18:31.618105 10061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.600845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.7.170.3:56227)
0929 02:18:31.605857 (+ 5012us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.605872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.605980 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.606286 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.606288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.606655 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:31.606916 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.606924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.608323 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.608327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.609548 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.609562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.610492 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.610499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.610738 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.611401 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:31.611428 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:31.612444 (+ 1016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:31.615311 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.615316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.615323 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.615896 (+ 573us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.616322 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:31.616327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:31.616331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:31.616472 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:31.616991 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:31.616997 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:31.617187 (+ 190us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.617345 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.617657 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4883,"thread_start_us":4638,"threads_started":1}
I20260929 02:18:31.618690 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 9944
I20260929 02:18:31.618852 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:31.619226 9593 ts_manager.cc:194] Registered new tserver with Master: 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:31.619328 7848 external_mini_cluster.cc:1428] Setting key 9eb3e131203d6a2b6019533514f89c2e
I20260929 02:18:31.620093 9593 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:35703
I20260929 02:18:31.622356 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:18:31 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:31.663829 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.651957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39366 (local address 127.7.170.62:36311)
0929 02:18:31.652177 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.652183 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.652413 (+ 230us) server_negotiation.cc:407] Connection header received
0929 02:18:31.652557 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.652562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.652637 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.652720 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:31.653986 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.654698 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.655803 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.656236 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.656321 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.659644 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:31.659649 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:31.659652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:31.659682 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:31.661846 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.662337 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.662340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.662343 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.662469 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:31.662731 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:31.662734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:31.662736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:31.662962 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:31.663120 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.663337 (+ 217us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.663512 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:18:31.667343 9594 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:31.674021 9594 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:31.678755 9801 heartbeater.cc:503] Master 127.7.170.62:36311 was elected leader, sending a full tablet report...
I20260929 02:18:31.694274 10092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.685089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32773 (local address 127.0.0.1:34740)
0929 02:18:31.686202 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.686250 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.686473 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.687884 (+ 1411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.687891 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.687927 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:31.688289 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.688302 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.690107 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.690115 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.691803 (+ 1688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.691814 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.693386 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.693393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.693580 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.693632 (+ 52us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.693728 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.693902 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":266,"threads_started":1}
I20260929 02:18:31.694921 10094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.686078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46205 (local address 127.0.0.1:47960)
0929 02:18:31.687127 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.687142 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.687242 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.688467 (+ 1225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.688472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.688498 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:31.689044 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.689060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.690681 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.690686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.692715 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.692731 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.694180 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.694186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.694367 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.694408 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.694495 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.694609 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":690,"threads_started":1}
I20260929 02:18:31.695076 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.685232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34740 (local address 127.7.170.2:32773)
0929 02:18:31.687289 (+ 2057us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.687300 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.687329 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:18:31.687426 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.687435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.687696 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.687870 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:31.688499 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.689965 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.691947 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.693192 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.694387 (+ 1195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.694575 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.694636 (+ 61us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.694794 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1893,"thread_start_us":2580,"threads_started":1}
I20260929 02:18:31.695669 10095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.686326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47960 (local address 127.7.170.1:46205)
0929 02:18:31.687693 (+ 1367us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.687701 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.687728 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:18:31.687864 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.687871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.688089 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.688267 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:31.689218 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.690549 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.692887 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.693971 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.694090 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.694280 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.694947 (+ 667us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.695338 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1236,"thread_start_us":1092,"threads_started":1}
I20260929 02:18:31.697335 10096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.686970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:35439 (local address 127.0.0.1:58408)
0929 02:18:31.689063 (+ 2093us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:31.689079 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:31.689181 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:31.689950 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:31.689954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:31.689976 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:31.690328 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.690339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.692187 (+ 1848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.692192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.694497 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:31.694510 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.696180 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.696186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:31.696343 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.696373 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:18:31.696469 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 02:18:31.697044 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1965,"thread_start_us":1765,"threads_started":1}
I20260929 02:18:31.697670 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:31.687620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58408 (local address 127.7.170.3:35439)
0929 02:18:31.688582 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:31.688590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:31.689277 (+ 687us) server_negotiation.cc:407] Connection header received
0929 02:18:31.689360 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:31.689366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:31.689567 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:31.689718 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:31.690513 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.692054 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.694656 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:31.696010 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:31.696129 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:31.696326 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:31.696526 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:18:31.696674 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":652,"threads_started":1}
I20260929 02:18:31.698676 9870 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 (DEFAULT_TABLE table=test-table [id=a945df22cd6d43b8a43ec066cd8313b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:31.699045 9731 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 (DEFAULT_TABLE table=test-table [id=a945df22cd6d43b8a43ec066cd8313b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:31.700080 9731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7d683d0b344a488972387e2b04b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:31.700109 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7d683d0b344a488972387e2b04b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:31.700778 10009 tablet_service.cc:1467] Processing CreateTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 (DEFAULT_TABLE table=test-table [id=a945df22cd6d43b8a43ec066cd8313b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:31.702199 10009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9e7d683d0b344a488972387e2b04b3c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:31.710671 10098 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Bootstrap starting.
I20260929 02:18:31.713778 10098 tablet_bootstrap.cc:654] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:31.713770 10099 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Bootstrap starting.
I20260929 02:18:31.715698 10098 log.cc:824] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:31.716017 10100 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Bootstrap starting.
I20260929 02:18:31.716908 10099 tablet_bootstrap.cc:654] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:31.718006 10099 log.cc:824] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:31.719004 10100 tablet_bootstrap.cc:654] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:31.719208 10098 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: No bootstrap required, opened a new log
I20260929 02:18:31.719477 10098 ts_tablet_manager.cc:1397] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260929 02:18:31.720032 10100 log.cc:824] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:31.720781 10099 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: No bootstrap required, opened a new log
I20260929 02:18:31.721071 10099 ts_tablet_manager.cc:1397] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260929 02:18:31.722306 10100 tablet_bootstrap.cc:492] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: No bootstrap required, opened a new log
I20260929 02:18:31.722522 10100 ts_tablet_manager.cc:1397] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 02:18:31.729475 10098 raft_consensus.cc:348] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.729904 10098 raft_consensus.cc:374] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:31.730038 10098 raft_consensus.cc:729] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Initialized, Role: FOLLOWER
I20260929 02:18:31.729823 10100 raft_consensus.cc:348] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.730199 10100 raft_consensus.cc:374] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:31.730288 10100 raft_consensus.cc:729] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a29061cee741b5920488c85553f3c6, State: Initialized, Role: FOLLOWER
I20260929 02:18:31.730803 10100 consensus_queue.cc:260] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.730696 10099 raft_consensus.cc:348] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.730836 10098 consensus_queue.cc:260] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.731063 10099 raft_consensus.cc:374] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:31.731154 10099 raft_consensus.cc:729] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:31.731815 10099 consensus_queue.cc:260] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:31.732509 10100 ts_tablet_manager.cc:1428] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260929 02:18:31.732849 10079 heartbeater.cc:503] Master 127.7.170.62:36311 was elected leader, sending a full tablet report...
I20260929 02:18:31.735797 9940 heartbeater.cc:503] Master 127.7.170.62:36311 was elected leader, sending a full tablet report...
I20260929 02:18:31.736922 10099 ts_tablet_manager.cc:1428] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260929 02:18:31.738261 10098 ts_tablet_manager.cc:1428] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Time spent starting tablet: real 0.019s user 0.013s sys 0.004s
W20260929 02:18:31.866335 10080 tablet.cc:2369] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:18:31.892226 9941 tablet.cc:2369] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:18:31.921864 9802 tablet.cc:2369] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:32.005410 10105 raft_consensus.cc:482] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:32.005688 10105 raft_consensus.cc:504] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.007314 10105 leader_election.cc:288] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.013764 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.007913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:33229 (local address 127.7.170.3:35439)
0929 02:18:32.008122 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.008129 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.008723 (+ 594us) server_negotiation.cc:407] Connection header received
0929 02:18:32.008987 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.008993 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.009058 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.009142 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:32.009751 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.010417 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.012148 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.012867 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.013015 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.013120 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.013397 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.013527 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:18:32.013772 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.007669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:45839 (local address 127.7.170.2:32773)
0929 02:18:32.007874 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.007880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.008689 (+ 809us) server_negotiation.cc:407] Connection header received
0929 02:18:32.008888 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.008895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.008987 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.009053 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:32.009614 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.010370 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.012150 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.012864 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.013028 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.013130 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.013416 (+ 286us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.013537 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:18:32.013942 10108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.007774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:35439 (local address 127.7.170.1:33229)
0929 02:18:32.008630 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.008647 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.008821 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.009184 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.009189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.009220 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:32.009598 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.009608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.010550 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.010554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.012000 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.012011 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.013113 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.013127 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.013255 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.013278 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.013352 (+ 74us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.013509 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":23,"spinlock_wait_cycles":25984,"thread_start_us":500,"threads_started":1}
I20260929 02:18:32.013980 10107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.007733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32773 (local address 127.7.170.1:45839)
0929 02:18:32.008550 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.008581 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.008735 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.009091 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.009096 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.009119 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:32.009460 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.009476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.010507 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.010512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.011998 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.012010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.013150 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.013157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.013269 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.013290 (+ 21us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.013371 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.013541 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":25,"thread_start_us":469,"threads_started":1}
I20260929 02:18:32.014950 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7d683d0b344a488972387e2b04b3c5" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" is_pre_election: true
I20260929 02:18:32.014952 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7d683d0b344a488972387e2b04b3c5" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6" is_pre_election: true
I20260929 02:18:32.015410 9890 raft_consensus.cc:2454] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 0.
I20260929 02:18:32.015404 10029 raft_consensus.cc:2454] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 0.
I20260929 02:18:32.016019 9684 leader_election.cc:302] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:32.016348 10105 raft_consensus.cc:2781] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:32.016477 10105 raft_consensus.cc:482] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:32.016572 10105 raft_consensus.cc:3037] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.018653 10105 raft_consensus.cc:504] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.019276 10105 leader_election.cc:288] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 election: Requested vote from peers 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.019579 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7d683d0b344a488972387e2b04b3c5" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa"
I20260929 02:18:32.019753 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9e7d683d0b344a488972387e2b04b3c5" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6"
I20260929 02:18:32.019789 9890 raft_consensus.cc:3037] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.019953 10029 raft_consensus.cc:3037] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.022869 9890 raft_consensus.cc:2454] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 1.
I20260929 02:18:32.022869 10029 raft_consensus.cc:2454] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 1.
I20260929 02:18:32.023240 9684 leader_election.cc:302] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:32.023516 10105 raft_consensus.cc:2781] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:32.023923 10105 raft_consensus.cc:686] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 1 LEADER]: Becoming Leader. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Running, Role: LEADER
I20260929 02:18:32.024390 10105 consensus_queue.cc:237] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.029556 9593 catalog_manager.cc:5495] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f6e0096fa304161b1ed7875335d514d (127.7.170.1). New cstate: current_term: 1 leader_uuid: "5f6e0096fa304161b1ed7875335d514d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } health_report { overall_health: UNKNOWN } } }
Sep 29 02:18:32 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648310, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:32.078902 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.067021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36311 (local address 127.0.0.1:57896)
0929 02:18:32.067510 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.067525 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.067646 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.067948 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.067952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.068644 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:32.068899 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.068907 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.069985 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.069990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.070976 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.070987 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.071616 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.071620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.071727 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.072282 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:32.072302 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:32.074373 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:32.076958 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:32.076970 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:32.076978 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:32.077315 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:32.077610 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:32.077613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:32.077615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:32.077739 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:32.078156 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:32.078161 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:32.078304 (+ 143us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.078432 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.078604 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":262,"threads_started":1}
I20260929 02:18:32.078955 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.067013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57896 (local address 127.7.170.62:36311)
0929 02:18:32.067154 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.067158 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.067606 (+ 448us) server_negotiation.cc:407] Connection header received
0929 02:18:32.067743 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.067748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.067801 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.067871 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:32.069073 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.069866 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.071084 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.071475 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.071556 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.074540 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:32.074544 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:32.074547 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:32.074574 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:32.076818 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:32.077396 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:32.077399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:32.077401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:32.077524 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:32.077824 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:32.077827 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:32.077829 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:32.078062 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:32.078158 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.078553 (+ 395us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.078695 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:18:32.083169 9593 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:57896:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:18:32.084138 9593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:32.090480 9731 tablet_service.cc:1467] Processing CreateTablet for tablet 592e123129634dffaa32bce241677ee6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee2335102f4532bfadaabcd47f928d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:32.090528 9870 tablet_service.cc:1467] Processing CreateTablet for tablet 592e123129634dffaa32bce241677ee6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee2335102f4532bfadaabcd47f928d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:32.091342 9731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592e123129634dffaa32bce241677ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.091348 9870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592e123129634dffaa32bce241677ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.096756 10098 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: Bootstrap starting.
I20260929 02:18:32.096804 10099 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: Bootstrap starting.
I20260929 02:18:32.097196 10009 tablet_service.cc:1467] Processing CreateTablet for tablet 592e123129634dffaa32bce241677ee6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=01ee2335102f4532bfadaabcd47f928d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:18:32.098105 10009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 592e123129634dffaa32bce241677ee6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.098274 10098 tablet_bootstrap.cc:654] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.098327 10099 tablet_bootstrap.cc:654] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.107754 10100 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: Bootstrap starting.
I20260929 02:18:32.108203 10098 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: No bootstrap required, opened a new log
I20260929 02:18:32.108294 10098 ts_tablet_manager.cc:1397] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.009s
I20260929 02:18:32.108842 10098 raft_consensus.cc:348] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.109097 10098 raft_consensus.cc:374] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.109177 10098 raft_consensus.cc:729] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.109462 10100 tablet_bootstrap.cc:654] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.109357 10098 consensus_queue.cc:260] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.109755 10105 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.109952 10105 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.110352 10098 ts_tablet_manager.cc:1428] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:18:32.110935 10099 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: No bootstrap required, opened a new log
I20260929 02:18:32.111088 10099 ts_tablet_manager.cc:1397] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.005s
I20260929 02:18:32.111831 10099 raft_consensus.cc:348] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.112095 10099 raft_consensus.cc:374] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.113230 10099 raft_consensus.cc:729] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.113525 10099 consensus_queue.cc:260] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.113957 10106 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.114302 10106 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.115653 10099 ts_tablet_manager.cc:1428] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260929 02:18:32.116266 10100 tablet_bootstrap.cc:492] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: No bootstrap required, opened a new log
I20260929 02:18:32.116401 10100 ts_tablet_manager.cc:1397] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:18:32.117154 10100 raft_consensus.cc:348] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.117393 10100 raft_consensus.cc:374] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.117483 10100 raft_consensus.cc:729] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a29061cee741b5920488c85553f3c6, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.117676 10100 consensus_queue.cc:260] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.118026 10104 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.118178 10100 ts_tablet_manager.cc:1428] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:18:32.118319 10104 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.296413 10106 raft_consensus.cc:482] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:32.296663 10106 raft_consensus.cc:504] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.298362 10106 leader_election.cc:288] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.307195 10097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.298781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:53969 (local address 127.7.170.3:35439)
0929 02:18:32.299028 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.299035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.299975 (+ 940us) server_negotiation.cc:407] Connection header received
0929 02:18:32.300117 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.300123 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.300198 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.300286 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:32.300976 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.302057 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.304879 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.306039 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.306160 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.306256 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.306807 (+ 551us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.306940 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260929 02:18:32.307219 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.299100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:52749 (local address 127.7.170.1:46205)
0929 02:18:32.300130 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.300136 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.300152 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:18:32.300210 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.300214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.300277 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.300365 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:32.301046 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.302075 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.304888 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.306128 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.306236 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.306323 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.306806 (+ 483us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.306939 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":820,"threads_started":1}
I20260929 02:18:32.307390 10121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.298639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46205 (local address 127.7.170.2:52749)
0929 02:18:32.299611 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.299633 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.299777 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.300395 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.300399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.300434 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:32.300857 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.300866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.302194 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.302200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.304742 (+ 2542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.304751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.306312 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.306318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.306511 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.306670 (+ 159us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.306770 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.306949 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":256,"thread_start_us":868,"threads_started":1}
I20260929 02:18:32.307416 10120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.298861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:35439 (local address 127.7.170.2:53969)
0929 02:18:32.299854 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.299873 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.299995 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.300305 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.300308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.300327 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:32.300788 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.300801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.302197 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.302201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.304737 (+ 2536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.304752 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.306260 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.306267 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.306495 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.306670 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.306770 (+ 100us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.306945 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"mutex_wait_us":187,"thread_start_us":401,"threads_started":1}
I20260929 02:18:32.308068 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592e123129634dffaa32bce241677ee6" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6" is_pre_election: true
I20260929 02:18:32.308297 10029 raft_consensus.cc:2454] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:32.308285 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592e123129634dffaa32bce241677ee6" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d" is_pre_election: true
I20260929 02:18:32.308681 9756 raft_consensus.cc:2454] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:32.308727 9824 leader_election.cc:302] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa; no voters:
I20260929 02:18:32.309151 10106 raft_consensus.cc:2781] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:32.309274 10106 raft_consensus.cc:482] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:32.309365 10106 raft_consensus.cc:3037] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.311498 10106 raft_consensus.cc:504] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.312036 10106 leader_election.cc:288] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Requested vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.312459 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592e123129634dffaa32bce241677ee6" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6"
I20260929 02:18:32.312469 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "592e123129634dffaa32bce241677ee6" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d"
I20260929 02:18:32.312635 9756 raft_consensus.cc:3037] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.312645 10029 raft_consensus.cc:3037] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.314502 10029 raft_consensus.cc:2454] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:32.314584 9756 raft_consensus.cc:2454] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:32.314850 9824 leader_election.cc:302] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa; no voters:
I20260929 02:18:32.315120 10106 raft_consensus.cc:2781] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:32.315574 10106 raft_consensus.cc:686] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 LEADER]: Becoming Leader. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Running, Role: LEADER
I20260929 02:18:32.316205 10106 consensus_queue.cc:237] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.319388 10123 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: TxnStatusTablet state changed. Reason: New leader 32d9596a1aeb46eabf7e374ccb58a1fa. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.319813 10123 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:32.320534 10125 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:18:32.320788 10125 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:18:32.321039 9593 catalog_manager.cc:5495] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2). New cstate: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:32.339435 10116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.333541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32773 (local address 127.0.0.1:33444)
0929 02:18:32.333759 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.333776 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.333884 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.334334 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.334337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.334346 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:32.334504 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.334510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.335929 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.335933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.336904 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.336915 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.337858 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.337862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.338006 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.338055 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.338987 (+ 932us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.338999 (+ 12us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.339051 (+ 52us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.339202 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:18:32.339481 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.333621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.7.170.2:32773)
0929 02:18:32.333791 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.333797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.333815 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:18:32.334065 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.334071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.334186 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.334328 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:32.334625 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.335792 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.337060 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.337720 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.337861 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.338221 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.338885 (+ 664us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.339019 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.339073 (+ 54us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.339154 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:18:32.346804 10029 raft_consensus.cc:1261] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.346869 9756 raft_consensus.cc:1261] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.347725 10123 consensus_queue.cc:1035] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.348026 10106 consensus_queue.cc:1035] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.357752 10105 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: TxnStatusTablet state changed. Reason: New leader 32d9596a1aeb46eabf7e374ccb58a1fa. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.358096 10105 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.362915 10106 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.365607 10106 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:32.365851 10128 mvcc.cc:205] Tried to move back new op lower bound from 7334495487363878912 to 7334495487250878464. Current Snapshot: MvccSnapshot[applied={T|T < 7334495487363878912 or (T in {7334495487363878912})}]
I20260929 02:18:32.368319 10104 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: TxnStatusTablet state changed. Reason: New leader 32d9596a1aeb46eabf7e374ccb58a1fa. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.368590 10104 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.372545 10105 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.372784 10105 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 5f6e0096fa304161b1ed7875335d514d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.383662 10104 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.383900 10104 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 03a29061cee741b5920488c85553f3c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:18:32.388410 10136 tablet_replica.cc:439] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } } }
I20260929 02:18:32.389421 10136 tablet_replica.cc:442] T 592e123129634dffaa32bce241677ee6 P 32d9596a1aeb46eabf7e374ccb58a1fa: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:18:32.399338 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.386244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54476 (local address 127.7.170.1:46205)
0929 02:18:32.386412 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.386418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.393503 (+ 7085us) server_negotiation.cc:407] Connection header received
0929 02:18:32.393708 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.393714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.393836 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.394016 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:32.394454 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.395565 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.397057 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.397681 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.397799 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.398145 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.398798 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.398938 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.398983 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.399063 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:18:32.415551 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.409614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:44065 (local address 127.7.170.2:32773)
0929 02:18:32.409797 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.409804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.410449 (+ 645us) server_negotiation.cc:407] Connection header received
0929 02:18:32.410652 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.410655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.410717 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.410784 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:32.411313 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.412020 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.413994 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.414408 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.414508 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.414988 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.415083 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.415141 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.415277 (+ 136us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.415338 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:18:32.415758 10143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.409579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32773 (local address 127.7.170.1:44065)
0929 02:18:32.410193 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.410212 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.410521 (+ 309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.410816 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.410821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.410834 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:32.411166 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.411177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.412132 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.412138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.413864 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.413878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.414573 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.414580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.414746 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.414812 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.415169 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.415180 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.415255 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.415387 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":328,"threads_started":1}
I20260929 02:18:32.424787 10029 raft_consensus.cc:1261] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Refusing update from remote peer 5f6e0096fa304161b1ed7875335d514d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.425022 9890 raft_consensus.cc:1261] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Refusing update from remote peer 5f6e0096fa304161b1ed7875335d514d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.425419 10109 consensus_queue.cc:1035] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.425843 10105 consensus_queue.cc:1035] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.432005 10129 mvcc.cc:205] Tried to move back new op lower bound from 7334495487686262784 to 7334495486055424000. Current Snapshot: MvccSnapshot[applied={T|T < 7334495487686262784}]
I20260929 02:18:32.457224 10122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.450154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56281 (local address 127.7.170.1:46205)
0929 02:18:32.450323 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:32.450329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:32.451386 (+ 1057us) server_negotiation.cc:407] Connection header received
0929 02:18:32.451600 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:32.451606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:32.451683 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:32.451765 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:32.452482 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.453280 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.455514 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.455981 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.456071 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.456608 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.456707 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.456778 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:32.456946 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 02:18:32.457018 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 02:18:32.457465 10147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:32.450173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46205 (local address 127.7.170.2:56281)
0929 02:18:32.451078 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:32.451141 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:32.451452 (+ 311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:32.451911 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:32.451917 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:32.451930 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:32.452292 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.452304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.453407 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.453413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.455295 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:32.455328 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:32.456157 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:32.456164 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:32.456345 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:32.456419 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:32.456805 (+ 386us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:32.456816 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:18:32.456896 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:18:32.457054 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":406,"threads_started":1}
I20260929 02:18:32.492758 9593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:32.493055 9593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:32.496878 9593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Sending DeleteTablet for 3 replicas of tablet 9e7d683d0b344a488972387e2b04b3c5
I20260929 02:18:32.498106 9731 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:32 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47960
I20260929 02:18:32.498574 10009 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:32 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:58408
I20260929 02:18:32.500742 9870 tablet_service.cc:1514] Processing DeleteTablet for tablet 9e7d683d0b344a488972387e2b04b3c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:32 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:34740
I20260929 02:18:32.509790 9593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:18:32.510665 10150 tablet_replica.cc:330] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: stopping tablet replica
W20260929 02:18:32.511083 9593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:32.511092 10150 raft_consensus.cc:2229] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:32.512259 10150 raft_consensus.cc:2258] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:32.512692 10151 tablet_replica.cc:330] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: stopping tablet replica
I20260929 02:18:32.513393 10152 tablet_replica.cc:330] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: stopping tablet replica
I20260929 02:18:32.513808 10152 raft_consensus.cc:2229] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:32.514734 10151 raft_consensus.cc:2229] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:32.515136 10151 raft_consensus.cc:2258] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:32.514139 10152 raft_consensus.cc:2258] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:32.518927 10152 ts_tablet_manager.cc:1905] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:32.519269 10151 ts_tablet_manager.cc:1905] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:32.519711 9731 tablet_service.cc:1467] Processing CreateTablet for tablet 2d6ac4010cd847af97c6066864f1d556 (DEFAULT_TABLE table=test-table [id=b88b3fda1ef94accaeebc59cf925527b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:32.519771 10008 tablet_service.cc:1467] Processing CreateTablet for tablet 2d6ac4010cd847af97c6066864f1d556 (DEFAULT_TABLE table=test-table [id=b88b3fda1ef94accaeebc59cf925527b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:32.520134 9731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d6ac4010cd847af97c6066864f1d556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.520208 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d6ac4010cd847af97c6066864f1d556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.520875 10150 ts_tablet_manager.cc:1905] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:32.526445 10100 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Bootstrap starting.
I20260929 02:18:32.528019 10100 tablet_bootstrap.cc:654] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.528075 10150 ts_tablet_manager.cc:1918] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:32.528234 10150 log.cc:1196] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/9e7d683d0b344a488972387e2b04b3c5
I20260929 02:18:32.528761 10150 ts_tablet_manager.cc:1939] T 9e7d683d0b344a488972387e2b04b3c5 P 5f6e0096fa304161b1ed7875335d514d: Deleting consensus metadata
I20260929 02:18:32.530426 9580 catalog_manager.cc:4841] TS 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205): tablet 9e7d683d0b344a488972387e2b04b3c5 (table test-table [id=a945df22cd6d43b8a43ec066cd8313b1]) successfully deleted
I20260929 02:18:32.531710 10098 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Bootstrap starting.
I20260929 02:18:32.533438 10098 tablet_bootstrap.cc:654] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.519562 9869 tablet_service.cc:1467] Processing CreateTablet for tablet 2d6ac4010cd847af97c6066864f1d556 (DEFAULT_TABLE table=test-table [id=b88b3fda1ef94accaeebc59cf925527b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:32.534615 9869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d6ac4010cd847af97c6066864f1d556. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:32.543031 10099 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Bootstrap starting.
I20260929 02:18:32.544579 10099 tablet_bootstrap.cc:654] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:32.559515 10100 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: No bootstrap required, opened a new log
I20260929 02:18:32.559819 10100 ts_tablet_manager.cc:1397] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Time spent bootstrapping tablet: real 0.033s user 0.010s sys 0.000s
I20260929 02:18:32.560679 10100 raft_consensus.cc:348] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.561100 10100 raft_consensus.cc:374] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.561230 10100 raft_consensus.cc:729] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a29061cee741b5920488c85553f3c6, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.561594 10100 consensus_queue.cc:260] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.561816 10099 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: No bootstrap required, opened a new log
I20260929 02:18:32.562021 10099 ts_tablet_manager.cc:1397] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.002s
I20260929 02:18:32.562795 10099 raft_consensus.cc:348] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.563060 10099 raft_consensus.cc:374] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.563104 10100 ts_tablet_manager.cc:1428] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:18:32.563166 10099 raft_consensus.cc:729] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.563699 10099 consensus_queue.cc:260] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.565207 10099 ts_tablet_manager.cc:1428] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260929 02:18:32.567102 10151 ts_tablet_manager.cc:1918] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:32.568574 10152 ts_tablet_manager.cc:1918] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:32.568794 10151 log.cc:1196] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/9e7d683d0b344a488972387e2b04b3c5
I20260929 02:18:32.569380 10151 ts_tablet_manager.cc:1939] T 9e7d683d0b344a488972387e2b04b3c5 P 03a29061cee741b5920488c85553f3c6: Deleting consensus metadata
I20260929 02:18:32.569437 10098 tablet_bootstrap.cc:492] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: No bootstrap required, opened a new log
I20260929 02:18:32.569640 10098 ts_tablet_manager.cc:1397] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Time spent bootstrapping tablet: real 0.038s user 0.009s sys 0.018s
I20260929 02:18:32.569795 10152 log.cc:1196] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/9e7d683d0b344a488972387e2b04b3c5
I20260929 02:18:32.570364 10152 ts_tablet_manager.cc:1939] T 9e7d683d0b344a488972387e2b04b3c5 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting consensus metadata
I20260929 02:18:32.570422 10098 raft_consensus.cc:348] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.570676 10098 raft_consensus.cc:374] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:32.570786 10098 raft_consensus.cc:729] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Initialized, Role: FOLLOWER
I20260929 02:18:32.571081 10098 consensus_queue.cc:260] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.572751 9581 catalog_manager.cc:4841] TS 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439): tablet 9e7d683d0b344a488972387e2b04b3c5 (table test-table [id=a945df22cd6d43b8a43ec066cd8313b1]) successfully deleted
I20260929 02:18:32.573999 9580 catalog_manager.cc:4841] TS 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773): tablet 9e7d683d0b344a488972387e2b04b3c5 (table test-table [id=a945df22cd6d43b8a43ec066cd8313b1]) successfully deleted
I20260929 02:18:32.574635 10098 ts_tablet_manager.cc:1428] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260929 02:18:32.690408 10105 raft_consensus.cc:482] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:32.690609 10105 raft_consensus.cc:504] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.691345 10105 leader_election.cc:288] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.691969 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d6ac4010cd847af97c6066864f1d556" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6" is_pre_election: true
I20260929 02:18:32.692214 10029 raft_consensus.cc:2454] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 0.
I20260929 02:18:32.692234 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d6ac4010cd847af97c6066864f1d556" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" is_pre_election: true
I20260929 02:18:32.692472 9890 raft_consensus.cc:2454] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 0.
I20260929 02:18:32.693004 9684 leader_election.cc:302] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:32.693301 10105 raft_consensus.cc:2781] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:32.693419 10105 raft_consensus.cc:482] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:32.693521 10105 raft_consensus.cc:3037] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.695600 10105 raft_consensus.cc:504] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.696602 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d6ac4010cd847af97c6066864f1d556" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6"
I20260929 02:18:32.696677 9890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d6ac4010cd847af97c6066864f1d556" candidate_uuid: "5f6e0096fa304161b1ed7875335d514d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa"
I20260929 02:18:32.696803 10029 raft_consensus.cc:3037] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.696856 9890 raft_consensus.cc:3037] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:32.699756 10029 raft_consensus.cc:2454] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 1.
I20260929 02:18:32.699807 9890 raft_consensus.cc:2454] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f6e0096fa304161b1ed7875335d514d in term 1.
I20260929 02:18:32.700271 9684 leader_election.cc:302] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:32.700455 10105 leader_election.cc:288] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [CANDIDATE]: Term 1 election: Requested vote from peers 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:32.700588 10109 raft_consensus.cc:2781] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:32.700771 10109 raft_consensus.cc:686] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 1 LEADER]: Becoming Leader. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Running, Role: LEADER
I20260929 02:18:32.701033 10109 consensus_queue.cc:237] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:32.703663 9593 catalog_manager.cc:5495] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f6e0096fa304161b1ed7875335d514d (127.7.170.1). New cstate: current_term: 1 leader_uuid: "5f6e0096fa304161b1ed7875335d514d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:32.753414 10029 raft_consensus.cc:1261] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Refusing update from remote peer 5f6e0096fa304161b1ed7875335d514d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.753973 9890 raft_consensus.cc:1261] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Refusing update from remote peer 5f6e0096fa304161b1ed7875335d514d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:32.754560 10109 consensus_queue.cc:1035] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Connected to new peer: Peer: permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.754976 10109 consensus_queue.cc:1035] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:32.769439 10130 mvcc.cc:205] Tried to move back new op lower bound from 7334495489031925760 to 7334495488825925632. Current Snapshot: MvccSnapshot[applied={T|T < 7334495489031925760}]
W20260929 02:18:32.791846 10129 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:18:34.375530 9593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:34.375744 9593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:34.379616 9593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Sending DeleteTablet for 3 replicas of tablet 2d6ac4010cd847af97c6066864f1d556
I20260929 02:18:34.380327 9731 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d6ac4010cd847af97c6066864f1d556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47960
I20260929 02:18:34.380396 9869 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d6ac4010cd847af97c6066864f1d556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:34740
I20260929 02:18:34.380532 10008 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d6ac4010cd847af97c6066864f1d556 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:58408
I20260929 02:18:34.393877 10183 tablet_replica.cc:330] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: stopping tablet replica
I20260929 02:18:34.394186 10183 raft_consensus.cc:2229] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:34.394677 10183 raft_consensus.cc:2258] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:34.395511 10182 tablet_replica.cc:330] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: stopping tablet replica
I20260929 02:18:34.395690 10182 raft_consensus.cc:2229] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:34.396450 10183 ts_tablet_manager.cc:1905] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.396508 10184 tablet_replica.cc:330] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: stopping tablet replica
I20260929 02:18:34.396772 10184 raft_consensus.cc:2229] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:34.397078 10184 raft_consensus.cc:2258] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:34.398488 10184 ts_tablet_manager.cc:1905] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.401175 10182 raft_consensus.cc:2258] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:34.402104 10182 ts_tablet_manager.cc:1905] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.402741 10183 ts_tablet_manager.cc:1918] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:34.402915 10183 log.cc:1196] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/2d6ac4010cd847af97c6066864f1d556
I20260929 02:18:34.403337 10183 ts_tablet_manager.cc:1939] T 2d6ac4010cd847af97c6066864f1d556 P 5f6e0096fa304161b1ed7875335d514d: Deleting consensus metadata
I20260929 02:18:34.404490 9580 catalog_manager.cc:4841] TS 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205): tablet 2d6ac4010cd847af97c6066864f1d556 (table test-table [id=b88b3fda1ef94accaeebc59cf925527b]) successfully deleted
I20260929 02:18:34.405009 10184 ts_tablet_manager.cc:1918] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:34.405130 10184 log.cc:1196] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/2d6ac4010cd847af97c6066864f1d556
I20260929 02:18:34.405534 10184 ts_tablet_manager.cc:1939] T 2d6ac4010cd847af97c6066864f1d556 P 03a29061cee741b5920488c85553f3c6: Deleting consensus metadata
I20260929 02:18:34.406489 9581 catalog_manager.cc:4841] TS 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439): tablet 2d6ac4010cd847af97c6066864f1d556 (table test-table [id=b88b3fda1ef94accaeebc59cf925527b]) successfully deleted
I20260929 02:18:34.408135 10182 ts_tablet_manager.cc:1918] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:34.408268 10182 log.cc:1196] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/2d6ac4010cd847af97c6066864f1d556
I20260929 02:18:34.408722 10182 ts_tablet_manager.cc:1939] T 2d6ac4010cd847af97c6066864f1d556 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting consensus metadata
I20260929 02:18:34.409739 9580 catalog_manager.cc:4841] TS 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773): tablet 2d6ac4010cd847af97c6066864f1d556 (table test-table [id=b88b3fda1ef94accaeebc59cf925527b]) successfully deleted
Sep 29 02:18:34 dist-test-slave-8cs2 krb5kdc[9549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:18:34 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:34.439375 10192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:34.425650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.7.170.62:36311)
0929 02:18:34.426179 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:34.426183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:34.426463 (+ 280us) server_negotiation.cc:407] Connection header received
0929 02:18:34.426605 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:34.426608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:34.426665 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:34.426756 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:34.427969 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.428770 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.430109 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.430504 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.430592 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:34.434452 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:34.434457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:34.434460 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:34.434491 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:34.437220 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:34.437756 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:34.437760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:34.437762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:34.437870 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:34.438185 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:34.438188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:34.438190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:34.438424 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:34.438655 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:34.438970 (+ 315us) server_negotiation.cc:299] Negotiation successful
0929 02:18:34.439126 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":258,"threads_started":1}
I20260929 02:18:34.442771 9593 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:34.443359 9593 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:34.449340 9869 tablet_service.cc:1467] Processing CreateTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 (DEFAULT_TABLE table=test-table [id=27a213fff3ed4d3da387cc82fab52ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.449362 9731 tablet_service.cc:1467] Processing CreateTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 (DEFAULT_TABLE table=test-table [id=27a213fff3ed4d3da387cc82fab52ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.449787 9731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361e045908ba4abeb47a9e2e8aa37a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.449790 9869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361e045908ba4abeb47a9e2e8aa37a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.449775 10008 tablet_service.cc:1467] Processing CreateTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 (DEFAULT_TABLE table=test-table [id=27a213fff3ed4d3da387cc82fab52ef5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.450167 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 361e045908ba4abeb47a9e2e8aa37a78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.455449 10193 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Bootstrap starting.
I20260929 02:18:34.456846 10194 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Bootstrap starting.
I20260929 02:18:34.457396 10195 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Bootstrap starting.
I20260929 02:18:34.457484 10193 tablet_bootstrap.cc:654] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.459036 10194 tablet_bootstrap.cc:654] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.459064 10195 tablet_bootstrap.cc:654] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.460986 10194 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: No bootstrap required, opened a new log
I20260929 02:18:34.461226 10194 ts_tablet_manager.cc:1397] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:18:34.461663 10193 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: No bootstrap required, opened a new log
I20260929 02:18:34.461763 10193 ts_tablet_manager.cc:1397] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260929 02:18:34.462368 10194 raft_consensus.cc:348] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.462586 10194 raft_consensus.cc:374] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.462669 10194 raft_consensus.cc:729] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a29061cee741b5920488c85553f3c6, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.462739 10195 tablet_bootstrap.cc:492] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: No bootstrap required, opened a new log
I20260929 02:18:34.462666 10193 raft_consensus.cc:348] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.462862 10195 ts_tablet_manager.cc:1397] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260929 02:18:34.462900 10193 raft_consensus.cc:374] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.462999 10193 raft_consensus.cc:729] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.463135 10194 consensus_queue.cc:260] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.463270 10193 consensus_queue.cc:260] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.463696 10195 raft_consensus.cc:348] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.463869 10193 ts_tablet_manager.cc:1428] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:18:34.463976 10195 raft_consensus.cc:374] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.464123 10195 raft_consensus.cc:729] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.464349 10195 consensus_queue.cc:260] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.464955 10195 ts_tablet_manager.cc:1428] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260929 02:18:34.472743 10194 ts_tablet_manager.cc:1428] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Time spent starting tablet: real 0.011s user 0.002s sys 0.009s
I20260929 02:18:34.486815 10176 raft_consensus.cc:482] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:34.487012 10176 raft_consensus.cc:504] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.487699 10176 leader_election.cc:288] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:34.488157 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361e045908ba4abeb47a9e2e8aa37a78" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d" is_pre_election: true
I20260929 02:18:34.488162 10027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361e045908ba4abeb47a9e2e8aa37a78" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6" is_pre_election: true
I20260929 02:18:34.488354 10027 raft_consensus.cc:2454] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:34.488372 9756 raft_consensus.cc:2454] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:34.488790 9823 leader_election.cc:302] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32d9596a1aeb46eabf7e374ccb58a1fa, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:34.489101 10176 raft_consensus.cc:2781] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:34.489215 10176 raft_consensus.cc:482] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:34.489307 10176 raft_consensus.cc:3037] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.491395 10176 raft_consensus.cc:504] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.491997 10176 leader_election.cc:288] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Requested vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:34.492477 10027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361e045908ba4abeb47a9e2e8aa37a78" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6"
I20260929 02:18:34.492548 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "361e045908ba4abeb47a9e2e8aa37a78" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d"
I20260929 02:18:34.492692 10027 raft_consensus.cc:3037] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.492733 9756 raft_consensus.cc:3037] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.495621 10027 raft_consensus.cc:2454] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:34.495621 9756 raft_consensus.cc:2454] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:34.496064 9824 leader_election.cc:302] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:34.496403 10176 raft_consensus.cc:2781] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:34.496536 10176 raft_consensus.cc:686] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 LEADER]: Becoming Leader. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Running, Role: LEADER
I20260929 02:18:34.496953 10176 consensus_queue.cc:237] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.499693 9593 catalog_manager.cc:5495] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2). New cstate: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:34.528995 10202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:34.524367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.7.170.2:32773)
0929 02:18:34.524987 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:34.524992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:34.525012 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:18:34.525058 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:34.525061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:34.525118 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:34.525196 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:34.525486 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.526231 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.527445 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.527834 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.527943 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:34.528288 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:34.528381 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:34.528442 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:34.528647 (+ 205us) server_negotiation.cc:299] Negotiation successful
0929 02:18:34.528722 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":353,"threads_started":1}
I20260929 02:18:34.541429 10202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:34.536348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42503 (local address 127.7.170.2:32773)
0929 02:18:34.536517 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:34.536522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:34.536778 (+ 256us) server_negotiation.cc:407] Connection header received
0929 02:18:34.536893 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:34.536897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:34.536982 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:34.537043 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:34.537405 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.538107 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.539661 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.540475 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.540561 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:34.540889 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:34.541010 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:34.541071 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:34.541163 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 02:18:34.541215 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":49}
I20260929 02:18:34.541543 10205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:34.536283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32773 (local address 127.7.170.2:42503)
0929 02:18:34.536695 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:34.536710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:34.536808 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:34.537068 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:34.537071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:34.537079 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:18:34.537281 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:34.537287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.538280 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.538284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:34.539552 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:34.539563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:34.540620 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:34.540625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:34.540744 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:34.540777 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:18:34.541079 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:18:34.541087 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:18:34.541142 (+ 55us) client_negotiation.cc:240] Negotiation successful
0929 02:18:34.541232 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":14,"thread_start_us":219,"threads_started":1}
I20260929 02:18:34.548894 10027 raft_consensus.cc:1261] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:34.549053 9756 raft_consensus.cc:1261] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:34.549573 10162 consensus_queue.cc:1035] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:34.549893 10176 consensus_queue.cc:1035] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:34.592658 9592 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:34.592854 9592 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:34.595666 9592 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Sending DeleteTablet for 3 replicas of tablet 361e045908ba4abeb47a9e2e8aa37a78
I20260929 02:18:34.596450 9731 tablet_service.cc:1514] Processing DeleteTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47960
I20260929 02:18:34.596549 10008 tablet_service.cc:1514] Processing DeleteTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:58408
I20260929 02:18:34.596638 9869 tablet_service.cc:1514] Processing DeleteTablet for tablet 361e045908ba4abeb47a9e2e8aa37a78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:34 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:34740
I20260929 02:18:34.596757 10183 tablet_replica.cc:330] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: stopping tablet replica
I20260929 02:18:34.596788 10184 tablet_replica.cc:330] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: stopping tablet replica
I20260929 02:18:34.596961 10183 raft_consensus.cc:2229] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:34.596971 10184 raft_consensus.cc:2229] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:34.597131 10183 raft_consensus.cc:2258] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:34.597131 10184 raft_consensus.cc:2258] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:34.597378 10182 tablet_replica.cc:330] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: stopping tablet replica
I20260929 02:18:34.597553 10182 raft_consensus.cc:2229] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:34.597762 9592 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:18:34.597996 10182 raft_consensus.cc:2258] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260929 02:18:34.598481 9592 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:34.598589 10184 ts_tablet_manager.cc:1905] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.599063 10182 ts_tablet_manager.cc:1905] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.603660 10183 ts_tablet_manager.cc:1905] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:34.605159 9731 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 (DEFAULT_TABLE table=test-table [id=0e124f4802c641fba35f46927e042d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.605530 10008 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 (DEFAULT_TABLE table=test-table [id=0e124f4802c641fba35f46927e042d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.605741 9731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf7bcc2cc84e0fa90f631606420c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.605911 10008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf7bcc2cc84e0fa90f631606420c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.609460 9869 tablet_service.cc:1467] Processing CreateTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 (DEFAULT_TABLE table=test-table [id=0e124f4802c641fba35f46927e042d75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:34.609824 9869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fdbf7bcc2cc84e0fa90f631606420c56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:34.611214 10183 ts_tablet_manager.cc:1918] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:34.611340 10183 log.cc:1196] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/361e045908ba4abeb47a9e2e8aa37a78
I20260929 02:18:34.611704 10183 ts_tablet_manager.cc:1939] T 361e045908ba4abeb47a9e2e8aa37a78 P 5f6e0096fa304161b1ed7875335d514d: Deleting consensus metadata
I20260929 02:18:34.612681 9580 catalog_manager.cc:4841] TS 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205): tablet 361e045908ba4abeb47a9e2e8aa37a78 (table test-table [id=27a213fff3ed4d3da387cc82fab52ef5]) successfully deleted
I20260929 02:18:34.615720 10194 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Bootstrap starting.
I20260929 02:18:34.617869 10194 tablet_bootstrap.cc:654] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.620056 10184 ts_tablet_manager.cc:1918] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:34.620373 10195 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Bootstrap starting.
I20260929 02:18:34.621114 10182 ts_tablet_manager.cc:1918] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:18:34.622071 10195 tablet_bootstrap.cc:654] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.622408 10182 log.cc:1196] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/361e045908ba4abeb47a9e2e8aa37a78
I20260929 02:18:34.623312 10182 ts_tablet_manager.cc:1939] T 361e045908ba4abeb47a9e2e8aa37a78 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting consensus metadata
I20260929 02:18:34.625167 9580 catalog_manager.cc:4841] TS 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773): tablet 361e045908ba4abeb47a9e2e8aa37a78 (table test-table [id=27a213fff3ed4d3da387cc82fab52ef5]) successfully deleted
I20260929 02:18:34.626332 10184 log.cc:1196] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/361e045908ba4abeb47a9e2e8aa37a78
I20260929 02:18:34.627817 10184 ts_tablet_manager.cc:1939] T 361e045908ba4abeb47a9e2e8aa37a78 P 03a29061cee741b5920488c85553f3c6: Deleting consensus metadata
I20260929 02:18:34.628145 10193 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Bootstrap starting.
I20260929 02:18:34.630160 10193 tablet_bootstrap.cc:654] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:34.639958 10193 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: No bootstrap required, opened a new log
I20260929 02:18:34.640173 10193 ts_tablet_manager.cc:1397] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.007s
I20260929 02:18:34.640167 9581 catalog_manager.cc:4841] TS 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439): tablet 361e045908ba4abeb47a9e2e8aa37a78 (table test-table [id=27a213fff3ed4d3da387cc82fab52ef5]) successfully deleted
I20260929 02:18:34.640981 10193 raft_consensus.cc:348] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.641222 10193 raft_consensus.cc:374] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.641295 10193 raft_consensus.cc:729] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.641498 10193 consensus_queue.cc:260] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.642854 10193 ts_tablet_manager.cc:1428] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260929 02:18:34.643867 10195 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: No bootstrap required, opened a new log
I20260929 02:18:34.644083 10195 ts_tablet_manager.cc:1397] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.003s
I20260929 02:18:34.644968 10195 raft_consensus.cc:348] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.646615 10195 raft_consensus.cc:374] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.646828 10195 raft_consensus.cc:729] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e0096fa304161b1ed7875335d514d, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.647235 10195 consensus_queue.cc:260] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.650151 10195 ts_tablet_manager.cc:1428] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260929 02:18:34.651432 10194 tablet_bootstrap.cc:492] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: No bootstrap required, opened a new log
I20260929 02:18:34.651587 10194 ts_tablet_manager.cc:1397] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Time spent bootstrapping tablet: real 0.036s user 0.012s sys 0.014s
I20260929 02:18:34.652431 10194 raft_consensus.cc:348] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.652658 10194 raft_consensus.cc:374] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:34.652766 10194 raft_consensus.cc:729] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03a29061cee741b5920488c85553f3c6, State: Initialized, Role: FOLLOWER
I20260929 02:18:34.653029 10194 consensus_queue.cc:260] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.653486 10194 ts_tablet_manager.cc:1428] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:18:34.819411 10176 raft_consensus.cc:482] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:34.819622 10176 raft_consensus.cc:504] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.820382 10176 leader_election.cc:288] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:34.820886 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf7bcc2cc84e0fa90f631606420c56" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d" is_pre_election: true
I20260929 02:18:34.820910 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf7bcc2cc84e0fa90f631606420c56" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6" is_pre_election: true
I20260929 02:18:34.821156 9756 raft_consensus.cc:2454] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:34.821197 10029 raft_consensus.cc:2454] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 0.
I20260929 02:18:34.821623 9823 leader_election.cc:302] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa, 5f6e0096fa304161b1ed7875335d514d; no voters:
I20260929 02:18:34.821902 10176 raft_consensus.cc:2781] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:34.822022 10176 raft_consensus.cc:482] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:34.822213 10176 raft_consensus.cc:3037] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.825799 10176 raft_consensus.cc:504] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.826447 10176 leader_election.cc:288] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Requested vote from peers 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205), 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439)
I20260929 02:18:34.826920 10029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf7bcc2cc84e0fa90f631606420c56" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03a29061cee741b5920488c85553f3c6"
I20260929 02:18:34.826913 9756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fdbf7bcc2cc84e0fa90f631606420c56" candidate_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e0096fa304161b1ed7875335d514d"
I20260929 02:18:34.827101 10029 raft_consensus.cc:3037] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.827112 9756 raft_consensus.cc:3037] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:34.829187 10029 raft_consensus.cc:2454] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:34.829635 9824 leader_election.cc:302] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03a29061cee741b5920488c85553f3c6, 32d9596a1aeb46eabf7e374ccb58a1fa; no voters:
I20260929 02:18:34.829895 10176 raft_consensus.cc:2781] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:34.830047 10176 raft_consensus.cc:686] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 LEADER]: Becoming Leader. State: Replica: 32d9596a1aeb46eabf7e374ccb58a1fa, State: Running, Role: LEADER
I20260929 02:18:34.830114 9756 raft_consensus.cc:2454] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32d9596a1aeb46eabf7e374ccb58a1fa in term 1.
I20260929 02:18:34.830591 10176 consensus_queue.cc:237] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } }
I20260929 02:18:34.833241 9592 catalog_manager.cc:5495] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa reported cstate change: term changed from 0 to 1, leader changed from <none> to 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2). New cstate: current_term: 1 leader_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32d9596a1aeb46eabf7e374ccb58a1fa" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:34.867300 10029 raft_consensus.cc:1261] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:34.867522 9755 raft_consensus.cc:1261] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Refusing update from remote peer 32d9596a1aeb46eabf7e374ccb58a1fa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:34.868220 10137 consensus_queue.cc:1035] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "03a29061cee741b5920488c85553f3c6" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:34.869208 10176 consensus_queue.cc:1035] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6e0096fa304161b1ed7875335d514d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 02:18:34.902128 10160 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260929 02:18:36.496254 9593 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:36.496490 9593 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:36.500878 9593 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 701a8f36869b44dbbd4eb2a92f3115b4: Sending DeleteTablet for 3 replicas of tablet fdbf7bcc2cc84e0fa90f631606420c56
I20260929 02:18:36.501701 9731 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47960
I20260929 02:18:36.501768 9869 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:34740
I20260929 02:18:36.501816 10008 tablet_service.cc:1514] Processing DeleteTablet for tablet fdbf7bcc2cc84e0fa90f631606420c56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:58408
I20260929 02:18:36.502415 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9665
I20260929 02:18:36.514489 10235 tablet_replica.cc:330] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: stopping tablet replica
I20260929 02:18:36.514758 10235 raft_consensus.cc:2229] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:36.515039 10235 raft_consensus.cc:2258] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:36.515869 10237 tablet_replica.cc:330] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: stopping tablet replica
I20260929 02:18:36.516124 10236 tablet_replica.cc:330] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: stopping tablet replica
I20260929 02:18:36.516160 10237 raft_consensus.cc:2229] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:36.516378 10236 raft_consensus.cc:2229] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:36.516589 10237 raft_consensus.cc:2258] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:36.516677 10235 ts_tablet_manager.cc:1905] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:36.516806 10236 raft_consensus.cc:2258] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:36.517869 10236 ts_tablet_manager.cc:1905] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:36.518005 10237 ts_tablet_manager.cc:1905] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:36.524125 10236 ts_tablet_manager.cc:1918] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:36.524253 10236 log.cc:1196] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/fdbf7bcc2cc84e0fa90f631606420c56
I20260929 02:18:36.524614 10236 ts_tablet_manager.cc:1939] T fdbf7bcc2cc84e0fa90f631606420c56 P 32d9596a1aeb46eabf7e374ccb58a1fa: Deleting consensus metadata
I20260929 02:18:36.524654 10235 ts_tablet_manager.cc:1918] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:36.524801 10235 log.cc:1196] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/fdbf7bcc2cc84e0fa90f631606420c56
I20260929 02:18:36.524838 10237 ts_tablet_manager.cc:1918] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260929 02:18:36.525024 10237 log.cc:1196] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/fdbf7bcc2cc84e0fa90f631606420c56
I20260929 02:18:36.525250 10235 ts_tablet_manager.cc:1939] T fdbf7bcc2cc84e0fa90f631606420c56 P 5f6e0096fa304161b1ed7875335d514d: Deleting consensus metadata
I20260929 02:18:36.525445 10237 ts_tablet_manager.cc:1939] T fdbf7bcc2cc84e0fa90f631606420c56 P 03a29061cee741b5920488c85553f3c6: Deleting consensus metadata
I20260929 02:18:36.525710 9580 catalog_manager.cc:4841] TS 32d9596a1aeb46eabf7e374ccb58a1fa (127.7.170.2:32773): tablet fdbf7bcc2cc84e0fa90f631606420c56 (table test-table [id=0e124f4802c641fba35f46927e042d75]) successfully deleted
I20260929 02:18:36.526254 9580 catalog_manager.cc:4841] TS 5f6e0096fa304161b1ed7875335d514d (127.7.170.1:46205): tablet fdbf7bcc2cc84e0fa90f631606420c56 (table test-table [id=0e124f4802c641fba35f46927e042d75]) successfully deleted
I20260929 02:18:36.526479 9581 catalog_manager.cc:4841] TS 03a29061cee741b5920488c85553f3c6 (127.7.170.3:35439): tablet fdbf7bcc2cc84e0fa90f631606420c56 (table test-table [id=0e124f4802c641fba35f46927e042d75]) successfully deleted
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:36.540992 10238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:36.503461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54482 (local address 127.7.170.1:46205)
0929 02:18:36.515601 (+ 12140us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:36.515607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:36.515634 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:18:36.515700 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:36.515706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:36.515788 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:36.517778 (+ 1990us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:36.518029 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.519707 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.521623 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.522375 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.522508 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:36.534804 (+ 12296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:36.534822 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:36.534839 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:36.534879 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:36.537848 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.538647 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.538655 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.538657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.538868 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.539343 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.539347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.539349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.539738 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:36.540025 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:36.540381 (+ 356us) server_negotiation.cc:299] Negotiation successful
0929 02:18:36.540599 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11990,"thread_start_us":11873,"threads_started":1}
W20260929 02:18:36.541648 9796 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54482
W20260929 02:18:36.542275 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9665: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:36.542467 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9665
I20260929 02:18:36.561427 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9805
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:36.575601 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:36.562194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.7.170.2:32773)
0929 02:18:36.562706 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:36.562710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:36.562725 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:18:36.562776 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:36.562780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:36.562839 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:36.562937 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:36.564050 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.564778 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.566053 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.566477 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.566574 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:36.570462 (+ 3888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:36.570477 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:36.570494 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:36.570532 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:36.573209 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.573896 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.573903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.573905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.574023 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.574317 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.574320 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.574322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.574668 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:36.574894 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:36.575131 (+ 237us) server_negotiation.cc:299] Negotiation successful
0929 02:18:36.575285 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":278,"threads_started":1}
W20260929 02:18:36.576272 9935 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33466
W20260929 02:18:36.577148 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9805: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:36.577296 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9805
I20260929 02:18:36.598263 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9944
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[9549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648314, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:36.612485 10241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:36.599017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40122 (local address 127.7.170.3:35439)
0929 02:18:36.599519 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:36.599523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:36.599538 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:18:36.599590 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:36.599593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:36.599664 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:36.599750 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:36.600988 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.601715 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.602920 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:36.603302 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:36.603397 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:36.607371 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:36.607387 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:36.607404 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:36.607436 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:36.610206 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.610811 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.610817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.610820 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.610976 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:36.611275 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:36.611278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:36.611280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:36.611623 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:36.611844 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:36.612018 (+ 174us) server_negotiation.cc:299] Negotiation successful
0929 02:18:36.612186 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":250,"threads_started":1}
W20260929 02:18:36.613196 10074 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40122
W20260929 02:18:36.613878 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9944: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:36.614054 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9944
I20260929 02:18:36.628228 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 9562
I20260929 02:18:36.629338 9634 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:36.744902 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 9562
2026-09-29T02:18:36Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8214 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[10245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[10245](info): set up 2 sockets
Sep 29 02:18:36 dist-test-slave-8cs2 krb5kdc[10245](info): commencing operation
krb5kdc: starting...
W20260929 02:18:37.843278 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.006s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260929 02:18:38.087193 7848 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.244s user 0.003s sys 0.003s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260929 02:18:38.329147 7848 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.242s user 0.001s sys 0.005s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260929 02:18:38.597149 7848 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.268s user 0.000s sys 0.004s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/test-user.keytab.
W20260929 02:18:39.397049 7848 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.800s user 0.003s sys 0.002s
Sep 29 02:18:39 dist-test-slave-8cs2 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
W20260929 02:18:40.682713 7848 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.170.62: real 1.238s user 0.000s sys 0.008s
I20260929 02:18:40.683701 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40503
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:40503
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:41.011482 10258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:41.011772 10258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:41.011890 10258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:41.019336 10258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:41.019438 10258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:41.019515 10258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:41.019589 10258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:41.019651 10258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:41.029589 10258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:40503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:41.030987 10258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:41.032305 10258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:41.043817 10263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:41.043913 10264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:41.043875 10266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:41.044142 10258 server_base.cc:1034] running on GCE node
I20260929 02:18:41.045142 10258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:41.046571 10258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:41.047775 10258 hybrid_clock.cc:648] HybridClock initialized: now 1790648321047725 us; error 45 us; skew 500 ppm
I20260929 02:18:41.048102 10258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:41 dist-test-slave-8cs2 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648321, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:41.051052 10258 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:41.052461 10258 webserver.cc:460] Webserver started at http://127.7.170.62:36021/ using document root <none> and password file <none>
I20260929 02:18:41.053143 10258 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:41.053249 10258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:41.053540 10258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:41.056391 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "491f34b99b0048e4939a0756339091ae"
format_stamp: "Formatted at 2026-09-29 02:18:41 on dist-test-slave-8cs2"
server_key: "6a8c9207b37a91945f17615340a76c9e"
server_key_iv: "14143f3d1d8336ced173aeb1e15d17d7"
server_key_version: "encryptionkey@0"
I20260929 02:18:41.057165 10258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "491f34b99b0048e4939a0756339091ae"
format_stamp: "Formatted at 2026-09-29 02:18:41 on dist-test-slave-8cs2"
server_key: "6a8c9207b37a91945f17615340a76c9e"
server_key_iv: "14143f3d1d8336ced173aeb1e15d17d7"
server_key_version: "encryptionkey@0"
I20260929 02:18:41.062734 10258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:18:41.065992 10273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:41.067273 10258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:18:41.067463 10258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "491f34b99b0048e4939a0756339091ae"
format_stamp: "Formatted at 2026-09-29 02:18:41 on dist-test-slave-8cs2"
server_key: "6a8c9207b37a91945f17615340a76c9e"
server_key_iv: "14143f3d1d8336ced173aeb1e15d17d7"
server_key_version: "encryptionkey@0"
I20260929 02:18:41.067629 10258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:41.093084 10258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:41.096957 10258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:41.097213 10258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:41.113456 10258 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:40503
I20260929 02:18:41.113480 10324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:40503 every 8 connection(s)
I20260929 02:18:41.114773 10258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:41.118358 10325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:41.121441 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10258
I20260929 02:18:41.121637 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:41.122398 7848 external_mini_cluster.cc:1428] Setting key 40a6b82d9950bbbe753d4b796a8d46b4
I20260929 02:18:41.130506 10325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae: Bootstrap starting.
I20260929 02:18:41.134038 10325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:41.135260 10325 log.cc:824] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:18:41 dist-test-slave-8cs2 krb5kdc[10245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648319, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:41.138317 10325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae: No bootstrap required, opened a new log
I20260929 02:18:41.145105 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:41.124339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33726 (local address 127.7.170.62:40503)
0929 02:18:41.125045 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:41.125057 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:41.125096 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:18:41.126709 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:41.126730 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:41.127141 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:41.127465 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:41.129117 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:41.130409 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:41.132085 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:41.132742 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:41.132860 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:41.137326 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:41.137337 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:41.137354 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:41.137403 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:41.141006 (+ 3603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:41.141602 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:41.141611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:41.141619 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:41.141810 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:41.142111 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:41.142116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:41.142120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:41.142802 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:41.143083 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:41.143333 (+ 250us) server_negotiation.cc:299] Negotiation successful
0929 02:18:41.143777 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":287,"threads_started":1}
I20260929 02:18:41.147742 10325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } }
I20260929 02:18:41.148057 10325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:41.148123 10325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491f34b99b0048e4939a0756339091ae, State: Initialized, Role: FOLLOWER
I20260929 02:18:41.148571 10325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } }
I20260929 02:18:41.148758 10325 raft_consensus.cc:388] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:41.148844 10325 raft_consensus.cc:482] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:41.148999 10325 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:41.190645 10325 raft_consensus.cc:504] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } }
I20260929 02:18:41.191246 10325 leader_election.cc:302] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 491f34b99b0048e4939a0756339091ae; no voters:
I20260929 02:18:41.191872 10325 leader_election.cc:288] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:41.192029 10330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:41.193084 10330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [term 1 LEADER]: Becoming Leader. State: Replica: 491f34b99b0048e4939a0756339091ae, State: Running, Role: LEADER
I20260929 02:18:41.194164 10325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:41.194029 10330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } }
I20260929 02:18:41.196897 10332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 491f34b99b0048e4939a0756339091ae. Latest consensus state: current_term: 1 leader_uuid: "491f34b99b0048e4939a0756339091ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } } }
I20260929 02:18:41.197006 10331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "491f34b99b0048e4939a0756339091ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491f34b99b0048e4939a0756339091ae" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40503 } } }
I20260929 02:18:41.197299 10332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:41.197342 10331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:41.198947 10338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:41.203231 10338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:41.211716 10338 catalog_manager.cc:1347] Generated new cluster ID: d12acbb7f69349529c829f5a933ec20c
I20260929 02:18:41.211823 10338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:41.233642 10338 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:41.234936 10338 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:41.240885 10338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 491f34b99b0048e4939a0756339091ae: Generated new TSK 0
I20260929 02:18:41.241689 10338 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
W20260929 02:18:41.931005 7848 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.7.170.1: real 0.673s user 0.001s sys 0.009s
I20260929 02:18:41.931952 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:42.231927 10351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:42.232203 10351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:42.232324 10351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:42.239255 10351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:42.239363 10351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:42.239509 10351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:42.248839 10351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:42.250018 10351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:42.251223 10351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:42.263098 10357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:42.263151 10356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:42.263151 10359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:42.263909 10351 server_base.cc:1034] running on GCE node
I20260929 02:18:42.264430 10351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:42.265329 10351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:42.266531 10351 hybrid_clock.cc:648] HybridClock initialized: now 1790648322266481 us; error 50 us; skew 500 ppm
I20260929 02:18:42.266824 10351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:42 dist-test-slave-8cs2 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:42.276324 10351 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:42.277741 10351 webserver.cc:460] Webserver started at http://127.7.170.1:40809/ using document root <none> and password file <none>
I20260929 02:18:42.278389 10351 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:42.278501 10351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:42.278787 10351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:42.281550 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "93d91fc4f5344cabaeec868c0c8e6cf5"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "bc7908ce101d6d3522b4ede89c067dc4"
server_key_iv: "dc76d485b7970fb9387eebc9ff4eadfb"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.282263 10351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "93d91fc4f5344cabaeec868c0c8e6cf5"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "bc7908ce101d6d3522b4ede89c067dc4"
server_key_iv: "dc76d485b7970fb9387eebc9ff4eadfb"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.287051 10351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 02:18:42.290258 10366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.291534 10351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:18:42.291676 10351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "93d91fc4f5344cabaeec868c0c8e6cf5"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "bc7908ce101d6d3522b4ede89c067dc4"
server_key_iv: "dc76d485b7970fb9387eebc9ff4eadfb"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.291837 10351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:42.317978 10351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:42.321725 10351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:42.321990 10351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:42.322867 10351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:42.324147 10351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:42.324223 10351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.324342 10351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:42.324391 10351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.353415 10351 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:36037
I20260929 02:18:42.353461 10478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:36037 every 8 connection(s)
I20260929 02:18:42.354630 10351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:42.357744 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10351
I20260929 02:18:42.357941 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:42.358709 7848 external_mini_cluster.cc:1428] Setting key 965322e43a37471f089ec7c2b62c57ee
Sep 29 02:18:42 dist-test-slave-8cs2 krb5kdc[10245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:42.375083 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:42.357005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:42209 (local address 127.7.170.62:40503)
0929 02:18:42.357421 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:42.357427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:42.358558 (+ 1131us) server_negotiation.cc:407] Connection header received
0929 02:18:42.360644 (+ 2086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:42.360650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:42.360713 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:42.360797 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:42.362740 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.363443 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.365441 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.365835 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.365925 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:42.369202 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:42.369208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:42.369212 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:42.369254 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:42.372087 (+ 2833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:42.372829 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:42.372835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:42.372838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:42.373006 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:42.373387 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:42.373392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:42.373396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:42.373730 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:42.373887 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:42.374501 (+ 614us) server_negotiation.cc:299] Negotiation successful
0929 02:18:42.374719 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":216,"threads_started":1}
I20260929 02:18:42.375716 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:42.357468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40503 (local address 127.7.170.1:42209)
0929 02:18:42.358365 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:42.358424 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:42.360359 (+ 1935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:42.361093 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:42.361105 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:42.361779 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:42.362560 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:42.362584 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.363595 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.363600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:42.365323 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:42.365341 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.366036 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.366042 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:42.366276 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:42.366938 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:42.366958 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:42.368984 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:42.372236 (+ 3252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:42.372245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:42.372259 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:42.372698 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:42.373099 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:42.373102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:42.373104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:42.373267 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:42.373854 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:42.373866 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:42.374140 (+ 274us) client_negotiation.cc:769] Sending connection context
0929 02:18:42.374347 (+ 207us) client_negotiation.cc:240] Negotiation successful
0929 02:18:42.374700 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":309,"threads_started":1}
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:42.377185 10479 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40503
I20260929 02:18:42.377455 10479 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:42.378098 10479 heartbeater.cc:511] Master 127.7.170.62:40503 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:42.380263 10290 ts_manager.cc:194] Registered new tserver with Master: 93d91fc4f5344cabaeec868c0c8e6cf5 (127.7.170.1:36037)
I20260929 02:18:42.382117 10290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:42209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:42.399874 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:42.699146 10485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:42.699405 10485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:42.699527 10485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:42.706564 10485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:42.706664 10485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:42.706810 10485 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:42.716188 10485 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:42.717420 10485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:42.718664 10485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:42.730566 10490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:42.730626 10493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:42.730603 10491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:42.731334 10485 server_base.cc:1034] running on GCE node
I20260929 02:18:42.731881 10485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:42.734359 10485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:42.735589 10485 hybrid_clock.cc:648] HybridClock initialized: now 1790648322735490 us; error 83 us; skew 500 ppm
I20260929 02:18:42.735931 10485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:42 dist-test-slave-8cs2 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:42.739629 10485 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:42.740909 10485 webserver.cc:460] Webserver started at http://127.7.170.2:38265/ using document root <none> and password file <none>
I20260929 02:18:42.741540 10485 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:42.741636 10485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:42.741873 10485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:42.744557 10485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "01969edae6f243d8a5a945a9307dd75e"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "5e1ffbecfd85839fc9071f41f98154db"
server_key_iv: "64359c009b597dcd0e7244afcc866eb6"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.745293 10485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "01969edae6f243d8a5a945a9307dd75e"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "5e1ffbecfd85839fc9071f41f98154db"
server_key_iv: "64359c009b597dcd0e7244afcc866eb6"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.749989 10485 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 02:18:42.753139 10500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.754341 10485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:42.754472 10485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "01969edae6f243d8a5a945a9307dd75e"
format_stamp: "Formatted at 2026-09-29 02:18:42 on dist-test-slave-8cs2"
server_key: "5e1ffbecfd85839fc9071f41f98154db"
server_key_iv: "64359c009b597dcd0e7244afcc866eb6"
server_key_version: "encryptionkey@0"
I20260929 02:18:42.754626 10485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:42.808192 10485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:42.811981 10485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:42.812207 10485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:42.813117 10485 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:42.814360 10485 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:42.814428 10485 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.814518 10485 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:42.814559 10485 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:42.843067 10485 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:40701
I20260929 02:18:42.843116 10612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:40701 every 8 connection(s)
I20260929 02:18:42.844254 10485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:42.844691 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10485
I20260929 02:18:42.844864 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:42.845451 7848 external_mini_cluster.cc:1428] Setting key 7435d1c6d7afa9b5e32d356bd3ab7ef1
Sep 29 02:18:42 dist-test-slave-8cs2 krb5kdc[10245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648322, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:42.865262 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:42.846801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:57789 (local address 127.7.170.62:40503)
0929 02:18:42.846999 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:42.847005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:42.848288 (+ 1283us) server_negotiation.cc:407] Connection header received
0929 02:18:42.850697 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:42.850701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:42.850766 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:42.850839 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:42.852701 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.853446 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.855248 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.855842 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.855959 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:42.859669 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:42.859674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:42.859678 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:42.859708 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:42.862293 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:42.863140 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:42.863144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:42.863147 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:42.863262 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:42.863715 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:42.863719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:42.863721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:42.864000 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:42.864101 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:42.864813 (+ 712us) server_negotiation.cc:299] Negotiation successful
0929 02:18:42.865035 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
I20260929 02:18:42.866245 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:42.847315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40503 (local address 127.7.170.2:57789)
0929 02:18:42.848115 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:42.848167 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:42.850389 (+ 2222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:42.851114 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:42.851127 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:42.851827 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:42.852550 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:42.852572 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.853590 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.853595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:42.855111 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:42.855123 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:42.856064 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:42.856071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:42.856267 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:42.856903 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:42.856949 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:42.859425 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:42.862434 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:42.862443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:42.862460 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:42.863020 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:42.863364 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:42.863368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:42.863372 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:42.863612 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:42.864116 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:42.864121 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:42.864460 (+ 339us) client_negotiation.cc:769] Sending connection context
0929 02:18:42.864700 (+ 240us) client_negotiation.cc:240] Negotiation successful
0929 02:18:42.865110 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":314,"threads_started":1}
I20260929 02:18:42.867573 10613 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40503
I20260929 02:18:42.867846 10613 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:42.868501 10613 heartbeater.cc:511] Master 127.7.170.62:40503 requested a full tablet report, sending...
I20260929 02:18:42.869859 10290 ts_manager.cc:194] Registered new tserver with Master: 01969edae6f243d8a5a945a9307dd75e (127.7.170.2:40701)
I20260929 02:18:42.870850 10290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:57789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:42.886169 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:41543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:43.194346 10618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:43.194633 10618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:43.194741 10618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:43.201889 10618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:43.201982 10618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:43.202095 10618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:43.211387 10618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40503
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:43.212630 10618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:43.213851 10618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:43.226244 10624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:43.226217 10623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:43.226375 10626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:43.226841 10618 server_base.cc:1034] running on GCE node
I20260929 02:18:43.227375 10618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:43.228195 10618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:43.229391 10618 hybrid_clock.cc:648] HybridClock initialized: now 1790648323229350 us; error 31 us; skew 500 ppm
I20260929 02:18:43.229676 10618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:43 dist-test-slave-8cs2 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648323, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:43.232513 10618 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:43.233817 10618 webserver.cc:460] Webserver started at http://127.7.170.3:33973/ using document root <none> and password file <none>
I20260929 02:18:43.234462 10618 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:43.234565 10618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:43.234843 10618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:43.237655 10618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "e25c0c5ed86c4be09202cd3eff95fe9f"
format_stamp: "Formatted at 2026-09-29 02:18:43 on dist-test-slave-8cs2"
server_key: "a520cfb290b9d74bf92380fc1139c5fa"
server_key_iv: "dd94c4cef390151063a0a1dcaea71b0a"
server_key_version: "encryptionkey@0"
I20260929 02:18:43.238406 10618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "e25c0c5ed86c4be09202cd3eff95fe9f"
format_stamp: "Formatted at 2026-09-29 02:18:43 on dist-test-slave-8cs2"
server_key: "a520cfb290b9d74bf92380fc1139c5fa"
server_key_iv: "dd94c4cef390151063a0a1dcaea71b0a"
server_key_version: "encryptionkey@0"
I20260929 02:18:43.243340 10618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260929 02:18:43.246516 10633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:43.247766 10618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:18:43.247908 10618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e25c0c5ed86c4be09202cd3eff95fe9f"
format_stamp: "Formatted at 2026-09-29 02:18:43 on dist-test-slave-8cs2"
server_key: "a520cfb290b9d74bf92380fc1139c5fa"
server_key_iv: "dd94c4cef390151063a0a1dcaea71b0a"
server_key_version: "encryptionkey@0"
I20260929 02:18:43.248080 10618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:43.264537 10618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:43.268075 10618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:43.268287 10618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:43.269126 10618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:43.270342 10618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:43.270429 10618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:43.270536 10618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:43.270579 10618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:43.298758 10618 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:41089
I20260929 02:18:43.298787 10745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:41089 every 8 connection(s)
I20260929 02:18:43.300019 10618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:43.301786 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10618
I20260929 02:18:43.302024 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:43.302551 7848 external_mini_cluster.cc:1428] Setting key 8f0ae598ba93fd61d309aad63b13efd0
Sep 29 02:18:43 dist-test-slave-8cs2 krb5kdc[10245](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648323, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:43.318558 10481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:43.302523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:49065 (local address 127.7.170.62:40503)
0929 02:18:43.302685 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:43.302689 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:43.304081 (+ 1392us) server_negotiation.cc:407] Connection header received
0929 02:18:43.306171 (+ 2090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:43.306175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:43.306247 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:43.306332 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:43.307661 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:43.308348 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:43.309879 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:43.310290 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:43.310396 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:43.313454 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:43.313458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:43.313462 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:43.313491 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:43.315694 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:43.316448 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:43.316452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:43.316456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:43.316607 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:43.317065 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:43.317070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:43.317072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:43.317421 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:43.317555 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:43.318164 (+ 609us) server_negotiation.cc:299] Negotiation successful
0929 02:18:43.318321 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:18:43.319408 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:43.303132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40503 (local address 127.7.170.3:49065)
0929 02:18:43.303917 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:43.303965 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:43.305925 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:43.306522 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:43.306530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:43.306992 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:43.307526 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:43.307542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:43.308477 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:43.308480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:43.309752 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:43.309764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:43.310457 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:43.310461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:43.310658 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:43.311250 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:43.311272 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:43.313274 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:43.315819 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:43.315827 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:43.315842 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:43.316320 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:43.316715 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:43.316720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:43.316724 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:43.316953 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:43.317543 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:43.317550 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:43.317867 (+ 317us) client_negotiation.cc:769] Sending connection context
0929 02:18:43.318082 (+ 215us) client_negotiation.cc:240] Negotiation successful
0929 02:18:43.318430 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":321,"threads_started":1}
I20260929 02:18:43.320657 10746 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40503
I20260929 02:18:43.320895 10746 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:43.321544 10746 heartbeater.cc:511] Master 127.7.170.62:40503 requested a full tablet report, sending...
I20260929 02:18:43.322759 10290 ts_manager.cc:194] Registered new tserver with Master: e25c0c5ed86c4be09202cd3eff95fe9f (127.7.170.3:41089)
I20260929 02:18:43.323609 10290 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:49065
I20260929 02:18:43.329443 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:18:43.355814 10481 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:18:43.353500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41914 (local address 127.7.170.62:40503)
0929 02:18:43.353652 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:43.353656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:43.354196 (+ 540us) server_negotiation.cc:407] Connection header received
0929 02:18:43.354358 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:43.354362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:43.354424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:43.354519 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:43.355601 (+ 1082us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41914: BlockingRecv error: recv got EOF from 127.0.0.1:41914 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:18:43.358660 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10351
W20260929 02:18:43.361029 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10351: Not authorized: Client connection negotiation failed: client connection to 127.7.170.1:36037: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:18:43.361192 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10351
I20260929 02:18:43.373955 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10485
W20260929 02:18:43.376295 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10485: Not authorized: Client connection negotiation failed: client connection to 127.7.170.2:40701: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:18:43.376449 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10485
I20260929 02:18:43.388334 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10618
W20260929 02:18:43.390812 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10618: Not authorized: Client connection negotiation failed: client connection to 127.7.170.3:41089: server requires authentication, but client does not have Kerberos credentials available
I20260929 02:18:43.390918 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10618
I20260929 02:18:43.402696 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10258
I20260929 02:18:43.403810 10320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:43.503762 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10258
2026-09-29T02:18:43Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (6755 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:43 dist-test-slave-8cs2 krb5kdc[10763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:43 dist-test-slave-8cs2 krb5kdc[10763](info): set up 2 sockets
Sep 29 02:18:43 dist-test-slave-8cs2 krb5kdc[10763](info): commencing operation
krb5kdc: starting...
W20260929 02:18:44.481500 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.941s user 0.011s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:44 dist-test-slave-8cs2 krb5kdc[10763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:44.652398 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33163
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:33163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:44.958354 10776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:44.958689 10776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:44.958827 10776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:44.969573 10776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:44.969702 10776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:44.969779 10776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:44.969859 10776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:44.969935 10776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:44.982923 10776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:33163
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:44.984123 10776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:44.985425 10776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:44.996141 10782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:44.996186 10781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:44.996212 10784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:44.997052 10776 server_base.cc:1034] running on GCE node
I20260929 02:18:44.997732 10776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:44.999002 10776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:45.000204 10776 hybrid_clock.cc:648] HybridClock initialized: now 1790648325000153 us; error 42 us; skew 500 ppm
I20260929 02:18:45.000519 10776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:44 dist-test-slave-8cs2 krb5kdc[10763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648324, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:45.003409 10776 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:45.004743 10776 webserver.cc:460] Webserver started at http://127.7.170.62:41471/ using document root <none> and password file <none>
I20260929 02:18:45.005391 10776 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:45.005486 10776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:45.005731 10776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:45.008535 10776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e5c4361ec96a421dac015662099fb5b0"
format_stamp: "Formatted at 2026-09-29 02:18:45 on dist-test-slave-8cs2"
server_key: "7fbe761ff8c75812c38cee7bebc9c290"
server_key_iv: "c0aefa3248e9c65cfc235dea568d30b4"
server_key_version: "encryptionkey@0"
I20260929 02:18:45.009276 10776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e5c4361ec96a421dac015662099fb5b0"
format_stamp: "Formatted at 2026-09-29 02:18:45 on dist-test-slave-8cs2"
server_key: "7fbe761ff8c75812c38cee7bebc9c290"
server_key_iv: "c0aefa3248e9c65cfc235dea568d30b4"
server_key_version: "encryptionkey@0"
I20260929 02:18:45.014014 10776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:18:45.017158 10791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:45.018379 10776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:18:45.018510 10776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e5c4361ec96a421dac015662099fb5b0"
format_stamp: "Formatted at 2026-09-29 02:18:45 on dist-test-slave-8cs2"
server_key: "7fbe761ff8c75812c38cee7bebc9c290"
server_key_iv: "c0aefa3248e9c65cfc235dea568d30b4"
server_key_version: "encryptionkey@0"
I20260929 02:18:45.018661 10776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:45.036033 10776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:45.039978 10776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:45.040195 10776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:45.056118 10776 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:33163
I20260929 02:18:45.056138 10842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:33163 every 8 connection(s)
I20260929 02:18:45.057366 10776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:45.057693 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10776
I20260929 02:18:45.057857 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:45.058678 7848 external_mini_cluster.cc:1428] Setting key 55945c35d2ed7238e9a6c451c1e3e8ba
I20260929 02:18:45.061547 10843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:18:45 dist-test-slave-8cs2 krb5kdc[10763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:45.073390 10843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0: Bootstrap starting.
I20260929 02:18:45.077131 10843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:45.078118 10843 log.cc:824] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:45.080018 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:45.060387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46974 (local address 127.7.170.62:33163)
0929 02:18:45.061129 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:45.061140 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:45.061183 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:18:45.062718 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:45.062740 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:45.063100 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:45.063391 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:45.065038 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:45.066336 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:45.068025 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:45.068656 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:45.068773 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:45.072806 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:45.072814 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:45.072827 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:45.072860 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:45.076311 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:45.076910 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:45.076919 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:45.076962 (+ 43us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:45.077130 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:45.077508 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:45.077513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:45.077517 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:45.078014 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:45.078273 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:45.078708 (+ 435us) server_negotiation.cc:299] Negotiation successful
0929 02:18:45.079100 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":318,"threads_started":1}
I20260929 02:18:45.080785 10843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0: No bootstrap required, opened a new log
I20260929 02:18:45.083631 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:18:45.090459 10843 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } }
I20260929 02:18:45.090958 10843 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:45.091046 10843 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5c4361ec96a421dac015662099fb5b0, State: Initialized, Role: FOLLOWER
I20260929 02:18:45.091639 10843 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } }
I20260929 02:18:45.091848 10843 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:45.091962 10843 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:45.092103 10843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:45.094125 10843 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } }
I20260929 02:18:45.094578 10843 leader_election.cc:302] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5c4361ec96a421dac015662099fb5b0; no voters:
I20260929 02:18:45.095155 10843 leader_election.cc:288] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:45.095256 10849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:45.096272 10849 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [term 1 LEADER]: Becoming Leader. State: Replica: e5c4361ec96a421dac015662099fb5b0, State: Running, Role: LEADER
I20260929 02:18:45.096822 10849 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } }
I20260929 02:18:45.097324 10843 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:45.099401 10851 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5c4361ec96a421dac015662099fb5b0. Latest consensus state: current_term: 1 leader_uuid: "e5c4361ec96a421dac015662099fb5b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } } }
I20260929 02:18:45.099731 10851 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:45.100242 10850 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5c4361ec96a421dac015662099fb5b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5c4361ec96a421dac015662099fb5b0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33163 } } }
I20260929 02:18:45.100518 10850 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:45.113763 10856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:45.119419 10856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:45.140019 10856 catalog_manager.cc:1347] Generated new cluster ID: a7a2da3914e649dca6df02d5c0521252
I20260929 02:18:45.140172 10856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:45.148524 10846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:45.117455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.7.170.62:33163)
0929 02:18:45.117629 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:45.117635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:45.139277 (+ 21642us) server_negotiation.cc:407] Connection header received
0929 02:18:45.139452 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:45.139457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:45.139542 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:45.139650 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:45.141573 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:45.142678 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:45.144913 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:45.145568 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:45.145651 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:45.146966 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:45.146972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:45.146977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:45.147731 (+ 754us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:45.147783 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:45.147816 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:45.147943 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:45.148021 (+ 78us) server_negotiation.cc:299] Negotiation successful
0929 02:18:45.148108 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":44}
I20260929 02:18:45.156710 10856 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:45.157882 10856 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:45.167060 10856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5c4361ec96a421dac015662099fb5b0: Generated new TSK 0
I20260929 02:18:45.167771 10856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:18:45.174494 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10776
I20260929 02:18:45.175613 10838 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:45.261674 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10776
2026-09-29T02:18:45Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1753 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:45 dist-test-slave-8cs2 krb5kdc[10877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:45 dist-test-slave-8cs2 krb5kdc[10877](info): set up 2 sockets
Sep 29 02:18:45 dist-test-slave-8cs2 krb5kdc[10877](info): commencing operation
krb5kdc: starting...
W20260929 02:18:46.249258 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.956s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:46 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:46.415760 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44815
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:44815
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:46.713142 10890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:46.713434 10890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:46.713568 10890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:46.720558 10890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:46.720681 10890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:46.720758 10890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:46.720829 10890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:46.720887 10890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:46.730669 10890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:44815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:46.732326 10890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:46.733722 10890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:46.745431 10896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:46.745463 10895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:46.745632 10898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:46.746184 10890 server_base.cc:1034] running on GCE node
I20260929 02:18:46.746869 10890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:46.748127 10890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:46.749313 10890 hybrid_clock.cc:648] HybridClock initialized: now 1790648326749264 us; error 36 us; skew 500 ppm
I20260929 02:18:46.749630 10890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:46 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648326, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:46.752403 10890 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:46.753753 10890 webserver.cc:460] Webserver started at http://127.7.170.62:33181/ using document root <none> and password file <none>
I20260929 02:18:46.754371 10890 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:46.754480 10890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:46.754745 10890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:46.757537 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "94844d43c5b34815b29d4984517a654e"
format_stamp: "Formatted at 2026-09-29 02:18:46 on dist-test-slave-8cs2"
server_key: "af88e928cc63e3bb0fd3e56aa7ec0964"
server_key_iv: "ada6a803300c819999a68fdffbb2bbd5"
server_key_version: "encryptionkey@0"
I20260929 02:18:46.758217 10890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "94844d43c5b34815b29d4984517a654e"
format_stamp: "Formatted at 2026-09-29 02:18:46 on dist-test-slave-8cs2"
server_key: "af88e928cc63e3bb0fd3e56aa7ec0964"
server_key_iv: "ada6a803300c819999a68fdffbb2bbd5"
server_key_version: "encryptionkey@0"
I20260929 02:18:46.763062 10890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:18:46.766340 10905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:46.767588 10890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:46.767738 10890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "94844d43c5b34815b29d4984517a654e"
format_stamp: "Formatted at 2026-09-29 02:18:46 on dist-test-slave-8cs2"
server_key: "af88e928cc63e3bb0fd3e56aa7ec0964"
server_key_iv: "ada6a803300c819999a68fdffbb2bbd5"
server_key_version: "encryptionkey@0"
I20260929 02:18:46.767911 10890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:46.784605 10890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:46.788545 10890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:46.788791 10890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:46.804570 10890 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:44815
I20260929 02:18:46.804590 10956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:44815 every 8 connection(s)
I20260929 02:18:46.805867 10890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:46.809428 10957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:46.810139 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10890
I20260929 02:18:46.810355 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:46.811124 7848 external_mini_cluster.cc:1428] Setting key 85a2c302e649c99125f9cf408dc6234e
I20260929 02:18:46.822512 10957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e: Bootstrap starting.
Sep 29 02:18:46 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:46.826052 10957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:46.827034 10957 log.cc:824] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:46.829811 10957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e: No bootstrap required, opened a new log
I20260929 02:18:46.832655 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:46.813084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55668 (local address 127.7.170.62:44815)
0929 02:18:46.813787 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:46.813800 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:46.813851 (+ 51us) server_negotiation.cc:407] Connection header received
0929 02:18:46.815391 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:46.815411 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:46.815789 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:46.816103 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:46.817637 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:46.818872 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:46.820562 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:46.821270 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:46.821404 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:46.825750 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:46.825758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:46.825771 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:46.825831 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:46.828897 (+ 3066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:46.829555 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:46.829561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:46.829566 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:46.829717 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:46.830063 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:46.830067 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:46.830069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:46.830569 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:46.830859 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:46.831232 (+ 373us) server_negotiation.cc:299] Negotiation successful
0929 02:18:46.831605 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":312,"threads_started":1}
I20260929 02:18:46.837731 10957 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } }
I20260929 02:18:46.838059 10957 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:46.838156 10957 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94844d43c5b34815b29d4984517a654e, State: Initialized, Role: FOLLOWER
I20260929 02:18:46.838724 10957 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } }
I20260929 02:18:46.838927 10957 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:46.839048 10957 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:46.839191 10957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:46.841272 10957 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } }
I20260929 02:18:46.841722 10957 leader_election.cc:302] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94844d43c5b34815b29d4984517a654e; no voters:
I20260929 02:18:46.842242 10957 leader_election.cc:288] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:46.842348 10962 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:46.842970 10962 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [term 1 LEADER]: Becoming Leader. State: Replica: 94844d43c5b34815b29d4984517a654e, State: Running, Role: LEADER
I20260929 02:18:46.843463 10962 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } }
I20260929 02:18:46.844115 10957 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:46.846515 10964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94844d43c5b34815b29d4984517a654e. Latest consensus state: current_term: 1 leader_uuid: "94844d43c5b34815b29d4984517a654e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } } }
I20260929 02:18:46.846801 10964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:46.847474 10963 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94844d43c5b34815b29d4984517a654e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94844d43c5b34815b29d4984517a654e" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44815 } } }
I20260929 02:18:46.847705 10963 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:46.849010 10969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:46.854012 10969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:46.862104 10969 catalog_manager.cc:1347] Generated new cluster ID: dcdea148c7c2426f84913e94639e15d7
I20260929 02:18:46.862200 10969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:46.880164 10969 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:46.881453 10969 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:46.893460 10969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94844d43c5b34815b29d4984517a654e: Generated new TSK 0
I20260929 02:18:46.894554 10969 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:46.942677 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:47.249974 10983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:47.250236 10983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:47.250362 10983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:47.257468 10983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:47.257575 10983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:47.257716 10983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:47.267344 10983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:47.268604 10983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:47.270006 10983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:47.282415 10991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:47.282442 10989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:47.282632 10988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:47.282893 10983 server_base.cc:1034] running on GCE node
I20260929 02:18:47.283627 10983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:47.284482 10983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:47.285670 10983 hybrid_clock.cc:648] HybridClock initialized: now 1790648327285618 us; error 45 us; skew 500 ppm
I20260929 02:18:47.285970 10983 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:47 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:47.295708 10983 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:47.297207 10983 webserver.cc:460] Webserver started at http://127.7.170.1:36249/ using document root <none> and password file <none>
I20260929 02:18:47.297881 10983 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:47.297997 10983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:47.298282 10983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:47.301203 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "2b01cc5b0510444f9502fb572f9a0957"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "8672fa98845187725072439e41da2f76"
server_key_iv: "813b83f0c468c462d46bcd6ab866fb60"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.301949 10983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "2b01cc5b0510444f9502fb572f9a0957"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "8672fa98845187725072439e41da2f76"
server_key_iv: "813b83f0c468c462d46bcd6ab866fb60"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.306900 10983 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:18:47.310272 10998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.311643 10983 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:47.311784 10983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "2b01cc5b0510444f9502fb572f9a0957"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "8672fa98845187725072439e41da2f76"
server_key_iv: "813b83f0c468c462d46bcd6ab866fb60"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.311960 10983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:47.330739 10983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:47.335044 10983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:47.335320 10983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:47.336264 10983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:47.337669 10983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:47.337744 10983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.337867 10983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:47.337916 10983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.366918 10983 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:41333
I20260929 02:18:47.366962 11110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:41333 every 8 connection(s)
I20260929 02:18:47.368121 10983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:47.370092 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 10983
I20260929 02:18:47.370267 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:47.370977 7848 external_mini_cluster.cc:1428] Setting key ac58d0b2ae7bad587a5869b46bf0055c
Sep 29 02:18:47 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:47.388664 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:47.370589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:35605 (local address 127.7.170.62:44815)
0929 02:18:47.371062 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:47.371069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:47.372069 (+ 1000us) server_negotiation.cc:407] Connection header received
0929 02:18:47.374306 (+ 2237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:47.374310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:47.374369 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:47.374454 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:47.376281 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.377034 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.379043 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.379627 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.379744 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:47.382873 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:47.382879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:47.382883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:47.382916 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:47.385775 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:47.386437 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:47.386443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:47.386448 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:47.386595 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:47.386962 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:47.386967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:47.386970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:47.387321 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:47.387481 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:47.388072 (+ 591us) server_negotiation.cc:299] Negotiation successful
0929 02:18:47.388286 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":224,"threads_started":1}
I20260929 02:18:47.389428 11114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:47.371205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44815 (local address 127.7.170.1:35605)
0929 02:18:47.371922 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:47.371976 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:47.374038 (+ 2062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:47.374725 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:47.374738 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:47.375414 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:47.376132 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:47.376153 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.377180 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.377186 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:47.378920 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:47.378935 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.379802 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.379806 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:47.379991 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:47.380605 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:47.380629 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:47.382665 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:47.385914 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:47.385921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:47.385934 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:47.386324 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:47.386693 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:47.386697 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:47.386699 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:47.386863 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:47.387433 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:47.387438 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:47.387733 (+ 295us) client_negotiation.cc:769] Sending connection context
0929 02:18:47.387953 (+ 220us) client_negotiation.cc:240] Negotiation successful
0929 02:18:47.388343 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":270,"threads_started":1}
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:47.390913 11111 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44815
I20260929 02:18:47.391186 11111 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:47.391902 11111 heartbeater.cc:511] Master 127.7.170.62:44815 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:47.394279 10922 ts_manager.cc:194] Registered new tserver with Master: 2b01cc5b0510444f9502fb572f9a0957 (127.7.170.1:41333)
I20260929 02:18:47.396229 10922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:35605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:47.419234 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:47.724774 11117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:47.725087 11117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:47.725219 11117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:47.732381 11117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:47.732497 11117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:47.732645 11117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:47.742093 11117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:47.743356 11117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:47.744639 11117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:47.756587 11125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:47.756784 11122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:47.756825 11123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:47.757684 11117 server_base.cc:1034] running on GCE node
I20260929 02:18:47.758279 11117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:47.759145 11117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:47.760321 11117 hybrid_clock.cc:648] HybridClock initialized: now 1790648327760264 us; error 54 us; skew 500 ppm
I20260929 02:18:47.760599 11117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:47 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:47.763497 11117 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:47.764818 11117 webserver.cc:460] Webserver started at http://127.7.170.2:43305/ using document root <none> and password file <none>
I20260929 02:18:47.765506 11117 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:47.765599 11117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:47.765849 11117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:47.768599 11117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "4e80be2a281f44778810fbbdaf0258bd"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "f1dcbfeaaa79b1f231d83013ba2965d2"
server_key_iv: "d5e1ad72d357cb1402360e4cc00a0b32"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.769326 11117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "4e80be2a281f44778810fbbdaf0258bd"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "f1dcbfeaaa79b1f231d83013ba2965d2"
server_key_iv: "d5e1ad72d357cb1402360e4cc00a0b32"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.774222 11117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:18:47.777403 11132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.778671 11117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:18:47.778803 11117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "4e80be2a281f44778810fbbdaf0258bd"
format_stamp: "Formatted at 2026-09-29 02:18:47 on dist-test-slave-8cs2"
server_key: "f1dcbfeaaa79b1f231d83013ba2965d2"
server_key_iv: "d5e1ad72d357cb1402360e4cc00a0b32"
server_key_version: "encryptionkey@0"
I20260929 02:18:47.778988 11117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:47.792168 11117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:47.795980 11117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:47.796229 11117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:47.797101 11117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:47.798292 11117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:47.798358 11117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.798463 11117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:47.798516 11117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:47.828624 11117 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44871
I20260929 02:18:47.828667 11244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44871 every 8 connection(s)
I20260929 02:18:47.829802 11117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:47.833823 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11117
I20260929 02:18:47.834028 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:47.834542 7848 external_mini_cluster.cc:1428] Setting key dbf695c080539bd81bf21a3990034ff8
Sep 29 02:18:47 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648327, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:47.850461 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:47.832355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:57629 (local address 127.7.170.62:44815)
0929 02:18:47.832529 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:47.832533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:47.833745 (+ 1212us) server_negotiation.cc:407] Connection header received
0929 02:18:47.835865 (+ 2120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:47.835870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:47.835947 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:47.836037 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:47.837911 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.838646 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.840698 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.841161 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.841259 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:47.844661 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:47.844668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:47.844672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:47.844724 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:47.847560 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:47.848260 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:47.848266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:47.848270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:47.848412 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:47.848794 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:47.848798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:47.848801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:47.849130 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:47.849277 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:47.849918 (+ 641us) server_negotiation.cc:299] Negotiation successful
0929 02:18:47.850135 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:18:47.851197 11247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:47.832815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44815 (local address 127.7.170.2:57629)
0929 02:18:47.833560 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:47.833616 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:47.835614 (+ 1998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:47.836264 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:47.836272 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:47.836974 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:47.837734 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:47.837758 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.838787 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.838792 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:47.840548 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:47.840562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:47.841355 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:47.841361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:47.841619 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:47.842251 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:47.842272 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:47.844460 (+ 2188us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:47.847697 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:47.847706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:47.847721 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:47.848158 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:47.848509 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:47.848513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:47.848515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:47.848707 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:47.849245 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:47.849250 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:47.849567 (+ 317us) client_negotiation.cc:769] Sending connection context
0929 02:18:47.849798 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 02:18:47.850150 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":265,"threads_started":1}
I20260929 02:18:47.852468 11245 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44815
I20260929 02:18:47.852787 11245 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:47.853603 11245 heartbeater.cc:511] Master 127.7.170.62:44815 requested a full tablet report, sending...
I20260929 02:18:47.854934 10922 ts_manager.cc:194] Registered new tserver with Master: 4e80be2a281f44778810fbbdaf0258bd (127.7.170.2:44871)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
I20260929 02:18:47.855762 10922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:57629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:47.875207 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:48.179440 11250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:48.179715 11250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:48.179852 11250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:48.186937 11250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:48.187057 11250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:48.187202 11250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:48.196678 11250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:44815
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:48.197908 11250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:48.199169 11250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:48.211184 11258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:48.211228 11256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:48.211252 11255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:48.212056 11250 server_base.cc:1034] running on GCE node
I20260929 02:18:48.212596 11250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:48.213471 11250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:48.214658 11250 hybrid_clock.cc:648] HybridClock initialized: now 1790648328214615 us; error 40 us; skew 500 ppm
I20260929 02:18:48.214982 11250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:48.217895 11250 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:48.219184 11250 webserver.cc:460] Webserver started at http://127.7.170.3:36729/ using document root <none> and password file <none>
I20260929 02:18:48.219816 11250 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:48.219916 11250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:48.220175 11250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:48.222859 11250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "ee672d008c13468cb49df95bfb212f61"
format_stamp: "Formatted at 2026-09-29 02:18:48 on dist-test-slave-8cs2"
server_key: "63dfab2b384deac4404d4ade33c37cff"
server_key_iv: "351e55896b2696b12bb8175a688729eb"
server_key_version: "encryptionkey@0"
I20260929 02:18:48.223569 11250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "ee672d008c13468cb49df95bfb212f61"
format_stamp: "Formatted at 2026-09-29 02:18:48 on dist-test-slave-8cs2"
server_key: "63dfab2b384deac4404d4ade33c37cff"
server_key_iv: "351e55896b2696b12bb8175a688729eb"
server_key_version: "encryptionkey@0"
I20260929 02:18:48.228279 11250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260929 02:18:48.231526 11265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:48.232769 11250 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:18:48.232903 11250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "ee672d008c13468cb49df95bfb212f61"
format_stamp: "Formatted at 2026-09-29 02:18:48 on dist-test-slave-8cs2"
server_key: "63dfab2b384deac4404d4ade33c37cff"
server_key_iv: "351e55896b2696b12bb8175a688729eb"
server_key_version: "encryptionkey@0"
I20260929 02:18:48.233114 11250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:48.266253 11250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:48.270017 11250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:48.270259 11250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:48.271183 11250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:48.272456 11250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:48.272537 11250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:48.272660 11250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:48.272706 11250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:48.301271 11250 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33285
I20260929 02:18:48.301324 11377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33285 every 8 connection(s)
I20260929 02:18:48.302484 11250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:48.309970 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11250
I20260929 02:18:48.310123 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:48.310598 7848 external_mini_cluster.cc:1428] Setting key 49f581011267c0ee6a6760f419e956d5
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:48.321272 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.304847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:48093 (local address 127.7.170.62:44815)
0929 02:18:48.305061 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:48.305067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:48.306257 (+ 1190us) server_negotiation.cc:407] Connection header received
0929 02:18:48.307974 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:48.307979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:48.308046 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:48.308130 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:48.309566 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.310393 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.312342 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.312832 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.312960 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.316339 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:48.316343 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:48.316347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:48.316378 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:48.318550 (+ 2172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.319187 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.319191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.319193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.319295 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.319675 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.319679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.319681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.319968 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:48.320089 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:48.320714 (+ 625us) server_negotiation.cc:299] Negotiation successful
0929 02:18:48.320950 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:18:48.322121 11380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.305423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44815 (local address 127.7.170.3:48093)
0929 02:18:48.306098 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:48.306140 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:48.307689 (+ 1549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:48.308327 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:48.308334 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:48.308807 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:48.309411 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:48.309428 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.310549 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.310555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:48.312185 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:48.312201 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.313100 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.313108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:48.313386 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.314040 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:48.314069 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:48.316154 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:48.318669 (+ 2515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:48.318678 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:48.318694 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:48.319076 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:48.319393 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:48.319398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:48.319402 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:48.319571 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:48.320080 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:48.320087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:48.320396 (+ 309us) client_negotiation.cc:769] Sending connection context
0929 02:18:48.320614 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 02:18:48.321129 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":275,"threads_started":1}
I20260929 02:18:48.323259 11378 heartbeater.cc:347] Connected to a master server at 127.7.170.62:44815
I20260929 02:18:48.323536 11378 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:48.324405 11378 heartbeater.cc:511] Master 127.7.170.62:44815 requested a full tablet report, sending...
I20260929 02:18:48.325829 10922 ts_manager.cc:194] Registered new tserver with Master: ee672d008c13468cb49df95bfb212f61 (127.7.170.3:33285)
I20260929 02:18:48.326587 10922 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:48093
I20260929 02:18:48.337456 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:48.380610 11113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.367737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55670 (local address 127.7.170.62:44815)
0929 02:18:48.367900 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:48.367905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:48.368371 (+ 466us) server_negotiation.cc:407] Connection header received
0929 02:18:48.368532 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:48.368536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:48.368598 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:48.368679 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:48.369973 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.370708 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.372102 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.372573 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.372680 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.376137 (+ 3457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:48.376142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:48.376145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:48.376178 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:48.378310 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.378871 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.378876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.378879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.378997 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.379339 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.379344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.379347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.379646 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:48.379850 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:48.380132 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:18:48.380333 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260929 02:18:48.381165 10922 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55670
I20260929 02:18:48.383260 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10983
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:48.396819 11389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.384041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37902 (local address 127.7.170.1:41333)
0929 02:18:48.384543 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:48.384549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:48.384572 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:18:48.384621 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:48.384627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:48.384820 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:48.384960 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:48.385923 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.386885 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.387981 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.388517 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.388608 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.392102 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:48.392112 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:48.392125 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:48.392155 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:48.394638 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.395188 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.395194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.395197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.395305 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.395585 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.395587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.395589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.395871 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:48.396091 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:48.396289 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 02:18:48.396467 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":239,"threads_started":1}
W20260929 02:18:48.398012 11106 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37902
W20260929 02:18:48.399008 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10983: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:48.399191 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10983
I20260929 02:18:48.399184 11111 heartbeater.cc:503] Master 127.7.170.62:44815 was elected leader, sending a full tablet report...
I20260929 02:18:48.411577 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11117
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:48.425431 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.412329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44592 (local address 127.7.170.2:44871)
0929 02:18:48.412840 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:48.412847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:48.412869 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:18:48.412950 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:48.412959 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:48.413164 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:48.413279 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:48.414336 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.415298 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.416407 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.416907 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.417095 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.420536 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:48.420546 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:48.420559 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:48.420588 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:48.423130 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.423634 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.423648 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.423650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.423780 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.424095 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.424098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.424100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.424421 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:48.424635 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:48.424859 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 02:18:48.425098 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":268,"threads_started":1}
W20260929 02:18:48.426633 11240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:44592
W20260929 02:18:48.427631 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11117: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:48.427747 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11117
I20260929 02:18:48.439909 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11250
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[10877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648328, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:48.454831 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:48.440706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52054 (local address 127.7.170.3:33285)
0929 02:18:48.441316 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:48.441323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:48.441350 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:18:48.441422 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:48.441428 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:48.441625 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:48.441769 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:48.443028 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.444068 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.445377 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:48.445971 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:48.446078 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:48.449992 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:48.450002 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:48.450016 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:48.450051 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:48.452461 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.453192 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.453200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.453202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.453335 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:48.453618 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:48.453621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:48.453623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:48.453957 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:48.454173 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:48.454371 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 02:18:48.454565 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":327,"threads_started":1}
W20260929 02:18:48.456063 11373 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:52054
W20260929 02:18:48.457103 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11250: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:18:48.457262 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11250
I20260929 02:18:48.469024 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 10890
I20260929 02:18:48.470049 10952 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:48.569118 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 10890
2026-09-29T02:18:48Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3312 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[11395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[11395](info): set up 2 sockets
Sep 29 02:18:48 dist-test-slave-8cs2 krb5kdc[11395](info): commencing operation
krb5kdc: starting...
W20260929 02:18:49.565277 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.959s user 0.000s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:49 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:49.743901 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33641
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:33641
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:50.041496 11408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:50.041745 11408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:50.041836 11408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:50.048599 11408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:50.048683 11408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:50.048736 11408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:50.048789 11408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:50.048837 11408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:50.058025 11408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:33641
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:50.059144 11408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:50.060319 11408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:50.070740 11413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:50.070778 11414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:50.070822 11416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:50.071702 11408 server_base.cc:1034] running on GCE node
I20260929 02:18:50.072355 11408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:50.073627 11408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:50.074816 11408 hybrid_clock.cc:648] HybridClock initialized: now 1790648330074771 us; error 35 us; skew 500 ppm
I20260929 02:18:50.075107 11408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:50 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:50.077912 11408 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:50.079188 11408 webserver.cc:460] Webserver started at http://127.7.170.62:35471/ using document root <none> and password file <none>
I20260929 02:18:50.079810 11408 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:50.079918 11408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:50.080192 11408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:50.082923 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "cba871af4e9e4661b86ac42ee85c9b17"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "d65664d8a75b2bb330941b489d766ed0"
server_key_iv: "1036f6a2d735349e72d94fde40015eab"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.083634 11408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "cba871af4e9e4661b86ac42ee85c9b17"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "d65664d8a75b2bb330941b489d766ed0"
server_key_iv: "1036f6a2d735349e72d94fde40015eab"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.088459 11408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:18:50.092264 11423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:50.093998 11408 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:18:50.094152 11408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "cba871af4e9e4661b86ac42ee85c9b17"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "d65664d8a75b2bb330941b489d766ed0"
server_key_iv: "1036f6a2d735349e72d94fde40015eab"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.094311 11408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:50.122231 11408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:50.126354 11408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:50.126595 11408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:50.143563 11408 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:33641
I20260929 02:18:50.143612 11474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:33641 every 8 connection(s)
I20260929 02:18:50.144866 11408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:50.148483 11475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:50.148612 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11408
I20260929 02:18:50.148823 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:50.149435 7848 external_mini_cluster.cc:1428] Setting key fc7c4ef28d7101991abe3162b75c44fa
I20260929 02:18:50.161131 11475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17: Bootstrap starting.
Sep 29 02:18:50 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:50.164680 11475 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:50.165733 11475 log.cc:824] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:50.168581 11475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17: No bootstrap required, opened a new log
I20260929 02:18:50.171890 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:50.151221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.7.170.62:33641)
0929 02:18:50.151897 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:50.151911 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:50.151949 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:18:50.153951 (+ 2002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:50.153972 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:50.154341 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:50.154656 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:50.156368 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.157670 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.159391 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.160013 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.160125 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:50.164524 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:50.164532 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:50.164546 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:50.164578 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:50.167744 (+ 3166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:50.168326 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:50.168335 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:50.168342 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:50.168532 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:50.168895 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:50.168900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:50.168903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:50.169601 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:50.169882 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:50.170172 (+ 290us) server_negotiation.cc:299] Negotiation successful
0929 02:18:50.170586 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":303,"threads_started":1}
I20260929 02:18:50.177126 11475 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } }
I20260929 02:18:50.177430 11475 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:50.177500 11475 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba871af4e9e4661b86ac42ee85c9b17, State: Initialized, Role: FOLLOWER
I20260929 02:18:50.177977 11475 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } }
I20260929 02:18:50.178186 11475 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:50.178334 11475 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:50.178506 11475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:50.180500 11475 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } }
I20260929 02:18:50.180968 11475 leader_election.cc:302] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba871af4e9e4661b86ac42ee85c9b17; no voters:
I20260929 02:18:50.181463 11475 leader_election.cc:288] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:50.181568 11480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:50.182353 11480 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [term 1 LEADER]: Becoming Leader. State: Replica: cba871af4e9e4661b86ac42ee85c9b17, State: Running, Role: LEADER
I20260929 02:18:50.182837 11480 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } }
I20260929 02:18:50.183313 11475 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:50.185422 11481 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba871af4e9e4661b86ac42ee85c9b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } } }
I20260929 02:18:50.185499 11482 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba871af4e9e4661b86ac42ee85c9b17. Latest consensus state: current_term: 1 leader_uuid: "cba871af4e9e4661b86ac42ee85c9b17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba871af4e9e4661b86ac42ee85c9b17" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33641 } } }
I20260929 02:18:50.185773 11481 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:50.185808 11482 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:50.190109 11486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:50.198458 11486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:50.206357 11486 catalog_manager.cc:1347] Generated new cluster ID: 9aef1f93a98e4854a83c4674d8e12a02
I20260929 02:18:50.206456 11486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:50.238736 11486 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:50.240070 11486 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:50.246459 11486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba871af4e9e4661b86ac42ee85c9b17: Generated new TSK 0
I20260929 02:18:50.247534 11486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:50.302788 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:50.604322 11501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:50.604619 11501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:50.604756 11501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:50.611956 11501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:50.612169 11501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:50.612352 11501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:50.621806 11501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:50.622992 11501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:50.624195 11501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:50.636224 11509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:50.636299 11506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:50.636264 11507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:50.636711 11501 server_base.cc:1034] running on GCE node
I20260929 02:18:50.637333 11501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:50.638159 11501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:50.639339 11501 hybrid_clock.cc:648] HybridClock initialized: now 1790648330639299 us; error 33 us; skew 500 ppm
I20260929 02:18:50.639670 11501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:50 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:50.642560 11501 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:50.643791 11501 webserver.cc:460] Webserver started at http://127.7.170.1:44955/ using document root <none> and password file <none>
I20260929 02:18:50.644377 11501 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:50.644479 11501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:50.644712 11501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:50.647451 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "7c1cb256de0f4bfbaa1ba2162018bb5e"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "0df9777ad6c811536dbf8ec9dcbcdef9"
server_key_iv: "3cfe8a7fb6b3b87711b9ecf3af1896f5"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.648109 11501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "7c1cb256de0f4bfbaa1ba2162018bb5e"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "0df9777ad6c811536dbf8ec9dcbcdef9"
server_key_iv: "3cfe8a7fb6b3b87711b9ecf3af1896f5"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.652653 11501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:18:50.655936 11516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:50.657172 11501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:50.657296 11501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "7c1cb256de0f4bfbaa1ba2162018bb5e"
format_stamp: "Formatted at 2026-09-29 02:18:50 on dist-test-slave-8cs2"
server_key: "0df9777ad6c811536dbf8ec9dcbcdef9"
server_key_iv: "3cfe8a7fb6b3b87711b9ecf3af1896f5"
server_key_version: "encryptionkey@0"
I20260929 02:18:50.657451 11501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:50.682883 11501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:50.686950 11501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:50.687180 11501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:50.688175 11501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:50.689545 11501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:50.689616 11501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:50.689703 11501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:50.689744 11501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:50.718742 11501 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:36479
I20260929 02:18:50.718786 11628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:36479 every 8 connection(s)
I20260929 02:18:50.719959 11501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:50.721661 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11501
I20260929 02:18:50.721863 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:50.722630 7848 external_mini_cluster.cc:1428] Setting key 27d35d50fce23b794795a4e3f696f4d3
Sep 29 02:18:50 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648330, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:50.743372 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:50.722429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:49841 (local address 127.7.170.62:33641)
0929 02:18:50.722837 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:50.722841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:50.723976 (+ 1135us) server_negotiation.cc:407] Connection header received
0929 02:18:50.725939 (+ 1963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:50.725945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:50.726015 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:50.726103 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:50.727488 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.728180 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.729891 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.730289 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.730402 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:50.736029 (+ 5627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:50.736035 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:50.736039 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:50.736074 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:50.740311 (+ 4237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:50.741236 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:50.741240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:50.741242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:50.741369 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:50.741870 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:50.741874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:50.741876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:50.742152 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:50.742259 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:50.742924 (+ 665us) server_negotiation.cc:299] Negotiation successful
0929 02:18:50.743107 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":200,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:50.744221 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:50.723066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33641 (local address 127.7.170.1:49841)
0929 02:18:50.723808 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:50.723854 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:50.725664 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:50.726335 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:50.726343 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:50.726808 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:50.727332 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:50.727348 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.728313 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.728317 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:50.729747 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:50.729761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:50.730493 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:50.730499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:50.730757 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:50.733116 (+ 2359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:50.733145 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:50.735806 (+ 2661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:50.740490 (+ 4684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:50.740499 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:50.740515 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:50.741105 (+ 590us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:50.741468 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:50.741473 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:50.741476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:50.741711 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:50.742263 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:50.742269 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:50.742576 (+ 307us) client_negotiation.cc:769] Sending connection context
0929 02:18:50.742809 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 02:18:50.743177 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":297,"threads_started":1}
I20260929 02:18:50.745740 11629 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33641
I20260929 02:18:50.746090 11629 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:50.746889 11629 heartbeater.cc:511] Master 127.7.170.62:33641 requested a full tablet report, sending...
I20260929 02:18:50.748965 11440 ts_manager.cc:194] Registered new tserver with Master: 7c1cb256de0f4bfbaa1ba2162018bb5e (127.7.170.1:36479)
I20260929 02:18:50.750679 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:49841
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:50.768055 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:51.073848 11635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:51.074095 11635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:51.074208 11635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:51.081197 11635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:51.081291 11635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:51.081459 11635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:51.090894 11635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:51.092110 11635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:51.093351 11635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:51.105875 11641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:51.105979 11640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:51.105952 11643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:51.106590 11635 server_base.cc:1034] running on GCE node
I20260929 02:18:51.107127 11635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:51.107926 11635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:51.109099 11635 hybrid_clock.cc:648] HybridClock initialized: now 1790648331109041 us; error 45 us; skew 500 ppm
I20260929 02:18:51.109400 11635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:51.112171 11635 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:51.113547 11635 webserver.cc:460] Webserver started at http://127.7.170.2:42267/ using document root <none> and password file <none>
I20260929 02:18:51.114153 11635 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:51.114261 11635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:51.114535 11635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:51.117283 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "a7a1160db69b4a10b1bd5caa3165739a"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "4b76608d1fd438647c225881485ef648"
server_key_iv: "07042999f889f3f5524437d7b5188a46"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.118005 11635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "a7a1160db69b4a10b1bd5caa3165739a"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "4b76608d1fd438647c225881485ef648"
server_key_iv: "07042999f889f3f5524437d7b5188a46"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.122850 11635 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:18:51.126070 11650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.127255 11635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:18:51.127408 11635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "a7a1160db69b4a10b1bd5caa3165739a"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "4b76608d1fd438647c225881485ef648"
server_key_iv: "07042999f889f3f5524437d7b5188a46"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.127578 11635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:51.141916 11635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:51.145777 11635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:51.146051 11635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:51.146981 11635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:51.148185 11635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:51.148257 11635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.148368 11635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:51.148430 11635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.177964 11635 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44227
I20260929 02:18:51.177999 11762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44227 every 8 connection(s)
I20260929 02:18:51.179181 11635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:51.183712 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11635
I20260929 02:18:51.183920 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:51.184306 7848 external_mini_cluster.cc:1428] Setting key 615c4aa735fe124e560872ab6274dc62
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:51.198421 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.181489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:46317 (local address 127.7.170.62:33641)
0929 02:18:51.181687 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.181693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.182821 (+ 1128us) server_negotiation.cc:407] Connection header received
0929 02:18:51.184704 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.184710 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.184794 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.184888 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.186493 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.187482 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.188978 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.189509 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.189625 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.192864 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.192870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.192875 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.192908 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.195294 (+ 2386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.196042 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.196046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.196048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.196171 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.196594 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.196597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.196598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.196828 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.196978 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.197907 (+ 929us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.198107 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:18:51.199653 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.181987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33641 (local address 127.7.170.2:46317)
0929 02:18:51.182657 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:51.182700 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:51.184394 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:51.185179 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:51.185187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:51.185650 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:51.186336 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:51.186354 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.187610 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.187614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:51.188815 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:51.188826 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.189684 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.189689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:51.189882 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.190522 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:51.190543 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:51.192654 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:51.195427 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:51.195435 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:51.195452 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:51.195934 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:51.196274 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:51.196278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:51.196282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:51.196503 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:51.196959 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:51.196966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:51.197365 (+ 399us) client_negotiation.cc:769] Sending connection context
0929 02:18:51.197722 (+ 357us) client_negotiation.cc:240] Negotiation successful
0929 02:18:51.198237 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":249,"threads_started":1}
I20260929 02:18:51.201079 11763 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33641
I20260929 02:18:51.201347 11763 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:51.202033 11763 heartbeater.cc:511] Master 127.7.170.62:33641 requested a full tablet report, sending...
I20260929 02:18:51.203408 11440 ts_manager.cc:194] Registered new tserver with Master: a7a1160db69b4a10b1bd5caa3165739a (127.7.170.2:44227)
I20260929 02:18:51.204195 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:46317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:51.226198 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:51.528807 11768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:51.529157 11768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:51.529294 11768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:51.536623 11768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:51.536767 11768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:51.536948 11768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:51.546692 11768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33641
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:51.547922 11768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:51.549254 11768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:51.564213 11773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:51.564256 11776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:51.564281 11774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:51.566123 11768 server_base.cc:1034] running on GCE node
I20260929 02:18:51.566850 11768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:51.567719 11768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:51.568920 11768 hybrid_clock.cc:648] HybridClock initialized: now 1790648331568858 us; error 53 us; skew 500 ppm
I20260929 02:18:51.569274 11768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:51.572279 11768 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:51.573781 11768 webserver.cc:460] Webserver started at http://127.7.170.3:42151/ using document root <none> and password file <none>
I20260929 02:18:51.574445 11768 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:51.574537 11768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:51.574788 11768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:51.577590 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "ad2a76cbecfc4d389180cf56e0cefbf9"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "5e96bf93b81a5b14e422227ef66d0aa8"
server_key_iv: "c9293e871a5c270b1665c07aeb16baf7"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.578331 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "ad2a76cbecfc4d389180cf56e0cefbf9"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "5e96bf93b81a5b14e422227ef66d0aa8"
server_key_iv: "c9293e871a5c270b1665c07aeb16baf7"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.583359 11768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:18:51.586768 11783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.588140 11768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:18:51.588290 11768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "ad2a76cbecfc4d389180cf56e0cefbf9"
format_stamp: "Formatted at 2026-09-29 02:18:51 on dist-test-slave-8cs2"
server_key: "5e96bf93b81a5b14e422227ef66d0aa8"
server_key_iv: "c9293e871a5c270b1665c07aeb16baf7"
server_key_version: "encryptionkey@0"
I20260929 02:18:51.588478 11768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:51.600378 11768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:51.604362 11768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:51.604630 11768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:51.605563 11768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:51.606875 11768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:51.606952 11768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.607043 11768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:51.607090 11768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:51.636189 11768 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36909
I20260929 02:18:51.636226 11895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36909 every 8 connection(s)
I20260929 02:18:51.637434 11768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:51.642030 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11768
I20260929 02:18:51.642221 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:51.642752 7848 external_mini_cluster.cc:1428] Setting key 74bc95b99230713ece080854dc472082
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:51.656242 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.639764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:40575 (local address 127.7.170.62:33641)
0929 02:18:51.639964 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.639978 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.641147 (+ 1169us) server_negotiation.cc:407] Connection header received
0929 02:18:51.643505 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.643511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.643595 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.643677 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.645294 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.646149 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.647673 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.648065 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.648167 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.651308 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.651313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.651316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.651347 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.653535 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.654185 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.654189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.654191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.654304 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.654682 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.654686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.654688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.654938 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.655060 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.655695 (+ 635us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.655909 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:18:51.657439 11898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.640239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33641 (local address 127.7.170.3:40575)
0929 02:18:51.640991 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:51.641041 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:51.643235 (+ 2194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:51.643922 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:51.643931 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:51.644535 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:51.645149 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:51.645168 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.646287 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.646291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:51.647543 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:51.647558 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.648261 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.648268 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:51.648493 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.649127 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:51.649148 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:51.651118 (+ 1970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:51.653653 (+ 2535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:51.653661 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:51.653678 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:51.654074 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:51.654397 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:51.654402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:51.654405 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:51.654574 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:51.655049 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:51.655056 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:51.655364 (+ 308us) client_negotiation.cc:769] Sending connection context
0929 02:18:51.655591 (+ 227us) client_negotiation.cc:240] Negotiation successful
0929 02:18:51.656148 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":251,"threads_started":1}
I20260929 02:18:51.658643 11896 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33641
I20260929 02:18:51.658932 11896 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:51.659564 11896 heartbeater.cc:511] Master 127.7.170.62:33641 requested a full tablet report, sending...
I20260929 02:18:51.660892 11440 ts_manager.cc:194] Registered new tserver with Master: ad2a76cbecfc4d389180cf56e0cefbf9 (127.7.170.3:36909)
I20260929 02:18:51.661723 11440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:40575
I20260929 02:18:51.670348 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:51.715898 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.701704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44736 (local address 127.7.170.1:36479)
0929 02:18:51.702277 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.702286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.702318 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:18:51.702390 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.702399 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.702627 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.702767 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.704048 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.705111 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.706300 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.706853 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.706974 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.710613 (+ 3639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.710624 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.710638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.710670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.713307 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.713919 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.713926 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.713928 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.714043 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.714418 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.714422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.714424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.714759 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.715043 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.715325 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.715580 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":285,"threads_started":1}
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:51.732261 11631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.719846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60550 (local address 127.7.170.62:33641)
0929 02:18:51.720044 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.720050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.720230 (+ 180us) server_negotiation.cc:407] Connection header received
0929 02:18:51.720374 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.720379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.720447 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.720527 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.721695 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.722389 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.723578 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.723961 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.724072 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.727714 (+ 3642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.727719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.727722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.727757 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.729987 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.730575 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.730579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.730582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.730708 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.731045 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.731049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.731051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.731358 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.731561 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.731875 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.732046 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:18:51.732757 11440 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60550
I20260929 02:18:51.733667 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11501
I20260929 02:18:51.744781 11903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.734426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.7.170.1:36479)
0929 02:18:51.734570 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.734574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.734842 (+ 268us) server_negotiation.cc:407] Connection header received
0929 02:18:51.734965 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.734969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.735056 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.735131 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.736237 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.736879 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.737990 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.738367 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.738452 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.740443 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.740448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.740451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.740495 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.742897 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.743425 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.743428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.743431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.743545 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.743801 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.743804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.743806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.744028 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.744179 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.744391 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.744551 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:18:51.745335 11624 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:51.824617 11629 heartbeater.cc:503] Master 127.7.170.62:33641 was elected leader, sending a full tablet report...
I20260929 02:18:51.825182 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11501
W20260929 02:18:51.838562 11426 connection.cc:535] server connection from 127.7.170.1:49841 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260929 02:18:51.838811 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11635
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:51.853379 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.839584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51368 (local address 127.7.170.2:44227)
0929 02:18:51.840093 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.840100 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.840126 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:18:51.840182 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.840189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.840412 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.840540 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.841656 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.842709 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.843923 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.844502 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.844601 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.848270 (+ 3669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.848281 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.848295 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.848327 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.850977 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.851535 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.851543 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.851545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.851678 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.852053 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.852056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.852058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.852377 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.852610 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.852857 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.853096 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":272,"threads_started":1}
I20260929 02:18:51.854667 11758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:51.935717 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11635
I20260929 02:18:51.947768 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11768
Sep 29 02:18:51 dist-test-slave-8cs2 krb5kdc[11395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:51.963400 11910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:51.948721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47488 (local address 127.7.170.3:36909)
0929 02:18:51.949351 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:51.949359 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:51.949389 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:18:51.949446 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:51.949452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:51.949662 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:51.949786 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:51.950945 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.952060 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.953632 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:51.954245 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:51.954343 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:51.958527 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:51.958544 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:51.958581 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:51.958631 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:51.961165 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.961761 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.961766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.961769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.961890 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:51.962153 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:51.962156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:51.962158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:51.962481 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:51.962708 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:51.962920 (+ 212us) server_negotiation.cc:299] Negotiation successful
0929 02:18:51.963118 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":308,"threads_started":1}
I20260929 02:18:51.964635 11891 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:52.044693 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11768
I20260929 02:18:52.056898 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11408
I20260929 02:18:52.058065 11470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:52.158970 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11408
2026-09-29T02:18:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3589 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:18:52 dist-test-slave-8cs2 krb5kdc[11915](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:18:52 dist-test-slave-8cs2 krb5kdc[11915](info): set up 2 sockets
Sep 29 02:18:52 dist-test-slave-8cs2 krb5kdc[11915](info): commencing operation
krb5kdc: starting...
W20260929 02:18:53.157403 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.962s user 0.001s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:18:53 dist-test-slave-8cs2 krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:18:53.335886 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36559
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:53.634210 11928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:53.634488 11928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:53.634608 11928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:53.641618 11928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:53.641716 11928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:53.641767 11928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:53.641848 11928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:53.641902 11928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:53.651633 11928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:53.652873 11928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:53.654165 11928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:53.665284 11934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:53.665321 11933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:53.665342 11936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:53.666028 11928 server_base.cc:1034] running on GCE node
I20260929 02:18:53.666790 11928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:53.668186 11928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:53.669402 11928 hybrid_clock.cc:648] HybridClock initialized: now 1790648333669335 us; error 43 us; skew 500 ppm
I20260929 02:18:53.669751 11928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:53 dist-test-slave-8cs2 krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:53.677685 11928 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:18:53.678876 11928 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:53.678968 11928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:53.679236 11928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:53.682293 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "faffcd17ecdc486e8307503def79e8e8"
format_stamp: "Formatted at 2026-09-29 02:18:53 on dist-test-slave-8cs2"
server_key: "f0f7ffbcb8d145e3ecd2a1145c3aaafb"
server_key_iv: "3c6459bd1703ebb7ae80e5b84596ec88"
server_key_version: "encryptionkey@0"
I20260929 02:18:53.683068 11928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "faffcd17ecdc486e8307503def79e8e8"
format_stamp: "Formatted at 2026-09-29 02:18:53 on dist-test-slave-8cs2"
server_key: "f0f7ffbcb8d145e3ecd2a1145c3aaafb"
server_key_iv: "3c6459bd1703ebb7ae80e5b84596ec88"
server_key_version: "encryptionkey@0"
I20260929 02:18:53.688275 11928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:18:53.691895 11942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:53.693306 11928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:18:53.693463 11928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "faffcd17ecdc486e8307503def79e8e8"
format_stamp: "Formatted at 2026-09-29 02:18:53 on dist-test-slave-8cs2"
server_key: "f0f7ffbcb8d145e3ecd2a1145c3aaafb"
server_key_iv: "3c6459bd1703ebb7ae80e5b84596ec88"
server_key_version: "encryptionkey@0"
I20260929 02:18:53.693662 11928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:53.707075 11928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:53.711028 11928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:53.711258 11928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:53.727584 11928 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:36559
I20260929 02:18:53.727605 11993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:36559 every 8 connection(s)
I20260929 02:18:53.728396 11928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:53.730759 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 11928
I20260929 02:18:53.730986 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:53.731607 7848 external_mini_cluster.cc:1428] Setting key daddd59692fb6fc9c6f88b3e761080d1
I20260929 02:18:53.732025 11994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:53.745492 11994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: Bootstrap starting.
Sep 29 02:18:53 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:53.749079 11994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:53.750363 11994 log.cc:824] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:53.753727 11994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: No bootstrap required, opened a new log
I20260929 02:18:53.755158 11997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:53.733530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58088 (local address 127.7.170.62:36559)
0929 02:18:53.734342 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:53.734357 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:53.734407 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:18:53.736589 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:53.736623 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:53.737146 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:53.737593 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:53.738873 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:53.740752 (+ 1879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:53.742000 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:53.743054 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:53.743240 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:53.746856 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:53.746867 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:53.746892 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:53.746951 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:53.750565 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:53.751432 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:53.751443 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:53.751449 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:53.751639 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:53.752056 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:53.752062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:53.752065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:53.752754 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:53.753050 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:53.753431 (+ 381us) server_negotiation.cc:299] Negotiation successful
0929 02:18:53.753873 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260929 02:18:53.761628 11994 raft_consensus.cc:348] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } }
I20260929 02:18:53.761960 11994 raft_consensus.cc:374] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:53.762032 11994 raft_consensus.cc:729] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faffcd17ecdc486e8307503def79e8e8, State: Initialized, Role: FOLLOWER
I20260929 02:18:53.762521 11994 consensus_queue.cc:260] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } }
I20260929 02:18:53.762710 11994 raft_consensus.cc:388] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:53.762799 11994 raft_consensus.cc:482] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:53.762918 11994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:53.765429 11994 raft_consensus.cc:504] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } }
I20260929 02:18:53.765877 11994 leader_election.cc:302] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faffcd17ecdc486e8307503def79e8e8; no voters:
I20260929 02:18:53.766415 11994 leader_election.cc:288] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:53.766561 11999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:53.767472 11999 raft_consensus.cc:686] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [term 1 LEADER]: Becoming Leader. State: Replica: faffcd17ecdc486e8307503def79e8e8, State: Running, Role: LEADER
I20260929 02:18:53.767937 11999 consensus_queue.cc:237] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } }
I20260929 02:18:53.768389 11994 sys_catalog.cc:564] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:53.770725 12001 sys_catalog.cc:455] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader faffcd17ecdc486e8307503def79e8e8. Latest consensus state: current_term: 1 leader_uuid: "faffcd17ecdc486e8307503def79e8e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } } }
I20260929 02:18:53.770756 12000 sys_catalog.cc:455] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faffcd17ecdc486e8307503def79e8e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faffcd17ecdc486e8307503def79e8e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36559 } } }
I20260929 02:18:53.771165 12000 sys_catalog.cc:458] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:53.771162 12001 sys_catalog.cc:458] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:53.773219 12007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:53.777213 12007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:53.786537 12007 catalog_manager.cc:1347] Generated new cluster ID: 56e5c75b870645b78a023ad05ee9b0b1
I20260929 02:18:53.786653 12007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:53.805243 12007 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:53.806613 12007 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:53.817543 12007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: Generated new TSK 0
I20260929 02:18:53.818361 12007 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:18:53.864756 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:54.170200 12020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:54.170465 12020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:54.170581 12020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:54.177670 12020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:54.177785 12020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:54.177932 12020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:54.187625 12020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:54.188869 12020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:54.190145 12020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:54.202099 12025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:54.202061 12028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:54.202121 12026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:54.202829 12020 server_base.cc:1034] running on GCE node
I20260929 02:18:54.203388 12020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:54.204231 12020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:54.205425 12020 hybrid_clock.cc:648] HybridClock initialized: now 1790648334205368 us; error 42 us; skew 500 ppm
I20260929 02:18:54.205714 12020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:54 dist-test-slave-8cs2 krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:54.208654 12020 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:18:54.209750 12020 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:54.209836 12020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:54.210090 12020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:54.212867 12020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "a356ffa348984ca6a337af19d9845b64"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "6c3238efa618dc82fb4a6eef72b1c829"
server_key_iv: "e26e149fbb3f10707a88c110aeb43b8e"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.213694 12020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "a356ffa348984ca6a337af19d9845b64"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "6c3238efa618dc82fb4a6eef72b1c829"
server_key_iv: "e26e149fbb3f10707a88c110aeb43b8e"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.218458 12020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:18:54.221696 12034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:54.222913 12020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:54.223028 12020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "a356ffa348984ca6a337af19d9845b64"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "6c3238efa618dc82fb4a6eef72b1c829"
server_key_iv: "e26e149fbb3f10707a88c110aeb43b8e"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.223189 12020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:54.245966 12020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:54.250137 12020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:54.250355 12020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:54.251338 12020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:54.252665 12020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:54.252748 12020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:54.252821 12020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:54.252880 12020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:54.282083 12020 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:38599
I20260929 02:18:54.282135 12146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:38599 every 8 connection(s)
I20260929 02:18:54.282888 12020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:54.283865 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12020
I20260929 02:18:54.284080 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:54.284853 7848 external_mini_cluster.cc:1428] Setting key 461812c58c32f6a8d16044c5589be203
Sep 29 02:18:54 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:54.304009 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:54.285540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:41939 (local address 127.7.170.62:36559)
0929 02:18:54.285987 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:54.285991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:54.287067 (+ 1076us) server_negotiation.cc:407] Connection header received
0929 02:18:54.289450 (+ 2383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:54.289454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:54.289513 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:54.289600 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:54.291463 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.292205 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.294053 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.294600 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.294715 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:54.297960 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:54.297966 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:54.297971 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:54.298006 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:54.300994 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:54.301671 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:54.301676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:54.301680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:54.301824 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:54.302274 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:54.302278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:54.302281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:54.302613 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:54.302758 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:54.303430 (+ 672us) server_negotiation.cc:299] Negotiation successful
0929 02:18:54.303637 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":230,"threads_started":1}
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:18:54.304802 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:54.286134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36559 (local address 127.7.170.1:41939)
0929 02:18:54.286893 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:54.286955 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:54.289180 (+ 2225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:54.289833 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:54.289844 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:54.290536 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:54.291290 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:54.291315 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.292338 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.292343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:54.293905 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:54.293919 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.294773 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.294777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:54.294957 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:54.295588 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:54.295609 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:54.297692 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:54.301115 (+ 3423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:54.301122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:54.301135 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:54.301545 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:54.301964 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:54.301968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:54.301970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:54.302161 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:54.302743 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:54.302749 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:54.303071 (+ 322us) client_negotiation.cc:769] Sending connection context
0929 02:18:54.303288 (+ 217us) client_negotiation.cc:240] Negotiation successful
0929 02:18:54.303659 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":293,"threads_started":1}
I20260929 02:18:54.306289 12147 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36559
I20260929 02:18:54.306629 12147 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:18:54.307467 12147 heartbeater.cc:511] Master 127.7.170.62:36559 requested a full tablet report, sending...
I20260929 02:18:54.309789 11959 ts_manager.cc:194] Registered new tserver with Master: a356ffa348984ca6a337af19d9845b64 (127.7.170.1:38599)
I20260929 02:18:54.311673 11959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:41939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:18:54.329193 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:54.635602 12153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:54.635896 12153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:54.636044 12153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:54.643250 12153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:54.643369 12153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:54.643523 12153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:54.653149 12153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:54.654428 12153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:54.655707 12153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:54.668135 12161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:54.668226 12158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:54.668237 12159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:54.669420 12153 server_base.cc:1034] running on GCE node
I20260929 02:18:54.670096 12153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:54.671022 12153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:54.672222 12153 hybrid_clock.cc:648] HybridClock initialized: now 1790648334672174 us; error 37 us; skew 500 ppm
I20260929 02:18:54.672525 12153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:54 dist-test-slave-8cs2 krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:54.675468 12153 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:18:54.676508 12153 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:54.676599 12153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:54.676918 12153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:54.679843 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "ea3f3df7c2274b139c84fce7737cbc56"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "c21daff3b57b2a1cdedb3c47907d6b5e"
server_key_iv: "73a23291cae3894f473d757ec3fb4ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.680613 12153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "ea3f3df7c2274b139c84fce7737cbc56"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "c21daff3b57b2a1cdedb3c47907d6b5e"
server_key_iv: "73a23291cae3894f473d757ec3fb4ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.685678 12153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260929 02:18:54.689111 12167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:54.690385 12153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:18:54.690529 12153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "ea3f3df7c2274b139c84fce7737cbc56"
format_stamp: "Formatted at 2026-09-29 02:18:54 on dist-test-slave-8cs2"
server_key: "c21daff3b57b2a1cdedb3c47907d6b5e"
server_key_iv: "73a23291cae3894f473d757ec3fb4ca2"
server_key_version: "encryptionkey@0"
I20260929 02:18:54.690724 12153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:54.730485 12153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:54.736236 12153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:54.736503 12153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:54.737682 12153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:54.739274 12153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:54.739418 12153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260929 02:18:54.739527 12153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:54.739599 12153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:54.769150 12153 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:42271
I20260929 02:18:54.769208 12279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:42271 every 8 connection(s)
I20260929 02:18:54.770020 12153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:54.775672 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12153
I20260929 02:18:54.775869 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:54.776412 7848 external_mini_cluster.cc:1428] Setting key e83785d99f510036f4f1166dba574174
Sep 29 02:18:54 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648334, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:54.790982 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:54.772314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42217 (local address 127.7.170.62:36559)
0929 02:18:54.772526 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:54.772532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:54.773768 (+ 1236us) server_negotiation.cc:407] Connection header received
0929 02:18:54.775554 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:54.775558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:54.775626 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:54.775700 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:54.777542 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.778611 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.780801 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.781223 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.781322 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:54.785162 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:54.785168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:54.785172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:54.785200 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:54.788038 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:54.788745 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:54.788751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:54.788754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:54.788899 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:54.789323 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:54.789329 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:54.789331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:54.789654 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:54.789785 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:54.790464 (+ 679us) server_negotiation.cc:299] Negotiation successful
0929 02:18:54.790674 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:18:54.791837 12282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:54.772840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36559 (local address 127.7.170.2:42217)
0929 02:18:54.773597 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:54.773635 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:54.775225 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:54.775978 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:54.775990 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:54.776670 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:54.777396 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:54.777415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.778768 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.778774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:54.780652 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:54.780667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:54.781445 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:54.781451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:54.781714 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:54.782398 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:54.782424 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:54.784854 (+ 2430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:54.788174 (+ 3320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:54.788181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:54.788195 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:54.788619 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:54.789025 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:54.789028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:54.789031 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:54.789206 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:54.789753 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:54.789759 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:54.790100 (+ 341us) client_negotiation.cc:769] Sending connection context
0929 02:18:54.790324 (+ 224us) client_negotiation.cc:240] Negotiation successful
0929 02:18:54.790717 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":277,"threads_started":1}
I20260929 02:18:54.793330 12280 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36559
I20260929 02:18:54.793669 12280 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:54.794528 12280 heartbeater.cc:511] Master 127.7.170.62:36559 requested a full tablet report, sending...
I20260929 02:18:54.796037 11959 ts_manager.cc:194] Registered new tserver with Master: ea3f3df7c2274b139c84fce7737cbc56 (127.7.170.2:42271)
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:18:54.796907 11959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:42217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:18:54.824893 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:18:55.146607 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:55.146870 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:55.146965 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:55.153995 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:55.154089 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:55.154203 12285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:55.164002 12285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:36559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:55.165230 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:55.166455 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:55.178390 12291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:55.178396 12293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:55.178982 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:55.179137 12285 server_base.cc:1034] running on GCE node
I20260929 02:18:55.179703 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:55.180527 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:55.181762 12285 hybrid_clock.cc:648] HybridClock initialized: now 1790648335181698 us; error 50 us; skew 500 ppm
I20260929 02:18:55.182070 12285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:18:55 dist-test-slave-8cs2 krb5kdc[11915](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648335, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:18:55.185242 12285 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:18:55.186329 12285 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:55.186426 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:55.186715 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:55.189464 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "27045ff8e8424b838f0ae40eaa07a7a8"
format_stamp: "Formatted at 2026-09-29 02:18:55 on dist-test-slave-8cs2"
server_key: "3c425127e7f9adb5e39c9e8d45222beb"
server_key_iv: "5854ced79fac1768dfe93b765e8efd6a"
server_key_version: "encryptionkey@0"
I20260929 02:18:55.190201 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "27045ff8e8424b838f0ae40eaa07a7a8"
format_stamp: "Formatted at 2026-09-29 02:18:55 on dist-test-slave-8cs2"
server_key: "3c425127e7f9adb5e39c9e8d45222beb"
server_key_iv: "5854ced79fac1768dfe93b765e8efd6a"
server_key_version: "encryptionkey@0"
I20260929 02:18:55.194918 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:18:55.198259 12299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:55.199653 12285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:55.199791 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "27045ff8e8424b838f0ae40eaa07a7a8"
format_stamp: "Formatted at 2026-09-29 02:18:55 on dist-test-slave-8cs2"
server_key: "3c425127e7f9adb5e39c9e8d45222beb"
server_key_iv: "5854ced79fac1768dfe93b765e8efd6a"
server_key_version: "encryptionkey@0"
I20260929 02:18:55.199966 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:55.214037 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:55.217931 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:55.218168 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:55.219038 12285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:55.220263 12285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:55.220353 12285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:55.220455 12285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:55.220532 12285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:55.247352 12285 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:42779
I20260929 02:18:55.247427 12411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:42779 every 8 connection(s)
I20260929 02:18:55.248225 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:55.252224 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12285
I20260929 02:18:55.252399 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:55.252768 7848 external_mini_cluster.cc:1428] Setting key 16687b0dcdd3879fc9b6b4a76f0801c1
Sep 29 02:18:55 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648335, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:18:55.266808 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.250484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:59963 (local address 127.7.170.62:36559)
0929 02:18:55.250632 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.250636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.251770 (+ 1134us) server_negotiation.cc:407] Connection header received
0929 02:18:55.253653 (+ 1883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.253659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.253739 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.253822 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:55.255366 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.256072 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.257588 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.257984 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.258088 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.261465 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:55.261470 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:55.261473 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:55.261505 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:55.263984 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.264660 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.264664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.264667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.264777 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.265223 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.265228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.265230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.265496 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:55.265621 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.266277 (+ 656us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.266483 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:18:55.267868 12414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.250956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36559 (local address 127.7.170.3:59963)
0929 02:18:55.251609 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.251651 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.253385 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.254075 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.254085 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.254676 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:55.255216 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.255233 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.256203 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.256207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.257463 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.257475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.258186 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.258193 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.258434 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.258974 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:18:55.258999 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:55.261250 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:55.264120 (+ 2870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:55.264129 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:55.264144 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:55.264547 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:55.264901 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:18:55.264907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:18:55.264910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:18:55.265116 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:18:55.265611 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:55.265619 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:55.265945 (+ 326us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.266167 (+ 222us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.266680 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":245,"threads_started":1}
I20260929 02:18:55.269181 12412 heartbeater.cc:347] Connected to a master server at 127.7.170.62:36559
I20260929 02:18:55.269434 12412 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:55.270048 12412 heartbeater.cc:511] Master 127.7.170.62:36559 requested a full tablet report, sending...
I20260929 02:18:55.271283 11959 ts_manager.cc:194] Registered new tserver with Master: 27045ff8e8424b838f0ae40eaa07a7a8 (127.7.170.3:42779)
I20260929 02:18:55.272017 11959 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:59963
I20260929 02:18:55.279716 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:18:55.295868 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.284291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.7.170.62:36559)
0929 02:18:55.284448 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.284454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.284991 (+ 537us) server_negotiation.cc:407] Connection header received
0929 02:18:55.285154 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.285158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.285225 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.285299 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:55.286866 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.287563 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.288782 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.289261 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.289364 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.291436 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:55.291441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:55.291444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:55.291474 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:55.293608 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.294159 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.294164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.294168 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.294284 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.294610 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.294616 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.294619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.294889 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:55.295117 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.295402 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.295607 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:18:55.299458 11959 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:55.306099 11959 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:55.314604 12147 heartbeater.cc:503] Master 127.7.170.62:36559 was elected leader, sending a full tablet report...
I20260929 02:18:55.338508 12425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.318063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53008 (local address 127.7.170.1:38599)
0929 02:18:55.330043 (+ 11980us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.330051 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.330824 (+ 773us) server_negotiation.cc:407] Connection header received
0929 02:18:55.331236 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.331242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.331436 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.331596 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:55.332189 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.333602 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.336064 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.337398 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.337538 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.337773 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.338018 (+ 245us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.338170 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11828,"thread_start_us":11695,"threads_started":1}
I20260929 02:18:55.338600 12426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.317875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:38599 (local address 127.0.0.1:53008)
0929 02:18:55.330490 (+ 12615us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.330746 (+ 256us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.331011 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.331628 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.331635 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.331664 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:55.332046 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.332056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.333712 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.333717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.335903 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.335917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.337583 (+ 1666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.337590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.337768 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.337818 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.337982 (+ 164us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.338160 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12344,"mutex_wait_us":3,"thread_start_us":12892,"threads_started":1}
I20260929 02:18:55.339092 12427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.318153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:42271 (local address 127.0.0.1:46912)
0929 02:18:55.330490 (+ 12337us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.330569 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.331229 (+ 660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.331770 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.331775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.331800 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:55.332260 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.332274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.334064 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.334069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.335752 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.335764 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.338371 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.338377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.338543 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.338573 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.338662 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.338772 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12179,"thread_start_us":11941,"threads_started":1}
I20260929 02:18:55.339290 12424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.318516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42779 (local address 127.0.0.1:49338)
0929 02:18:55.330881 (+ 12365us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.330895 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.331077 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.332215 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.332222 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.332250 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:55.332610 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.332623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.334851 (+ 2228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.334856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.336580 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.336590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.338493 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.338499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.338661 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.338691 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.338778 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.338898 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12240,"mutex_wait_us":5,"thread_start_us":11689,"threads_started":1}
I20260929 02:18:55.339279 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.319066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49338 (local address 127.7.170.3:42779)
0929 02:18:55.327279 (+ 8213us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.327289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.331005 (+ 3716us) server_negotiation.cc:407] Connection header received
0929 02:18:55.331838 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.331845 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.332040 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.332209 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:55.332805 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.334710 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.336759 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.338041 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.338150 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.338347 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.338818 (+ 471us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.338989 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8048,"thread_start_us":7903,"threads_started":1}
I20260929 02:18:55.339784 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.318644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46912 (local address 127.7.170.2:42271)
0929 02:18:55.329440 (+ 10796us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.329450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.330691 (+ 1241us) server_negotiation.cc:407] Connection header received
0929 02:18:55.331377 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.331383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.331576 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.331762 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:55.332480 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.333919 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.335894 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.337924 (+ 2030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.338047 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.338232 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.339328 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.339491 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10622,"thread_start_us":10800,"threads_started":1}
I20260929 02:18:55.342865 12215 tablet_service.cc:1467] Processing CreateTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (DEFAULT_TABLE table=test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:55.343039 12082 tablet_service.cc:1467] Processing CreateTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (DEFAULT_TABLE table=test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:55.343549 12347 tablet_service.cc:1467] Processing CreateTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (DEFAULT_TABLE table=test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:55.344175 12215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d572f6650e4d9e8ab36b2dcb2628d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:55.344446 12082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d572f6650e4d9e8ab36b2dcb2628d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:55.344798 12347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2d572f6650e4d9e8ab36b2dcb2628d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:55.355643 12430 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Bootstrap starting.
I20260929 02:18:55.358542 12431 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Bootstrap starting.
I20260929 02:18:55.359364 12430 tablet_bootstrap.cc:654] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:55.360585 12430 log.cc:824] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:55.361835 12431 tablet_bootstrap.cc:654] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:55.363221 12431 log.cc:824] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:55.364749 12432 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Bootstrap starting.
I20260929 02:18:55.366338 12431 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: No bootstrap required, opened a new log
I20260929 02:18:55.366606 12431 ts_tablet_manager.cc:1397] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260929 02:18:55.367849 12430 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: No bootstrap required, opened a new log
I20260929 02:18:55.368091 12430 ts_tablet_manager.cc:1397] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260929 02:18:55.368261 12432 tablet_bootstrap.cc:654] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:55.369467 12432 log.cc:824] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:55.375427 12430 raft_consensus.cc:348] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.375751 12430 raft_consensus.cc:374] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:55.375909 12430 raft_consensus.cc:729] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea3f3df7c2274b139c84fce7737cbc56, State: Initialized, Role: FOLLOWER
I20260929 02:18:55.376621 12430 consensus_queue.cc:260] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.376837 12431 raft_consensus.cc:348] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.377358 12431 raft_consensus.cc:374] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:55.377490 12431 raft_consensus.cc:729] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a356ffa348984ca6a337af19d9845b64, State: Initialized, Role: FOLLOWER
I20260929 02:18:55.377734 12280 heartbeater.cc:503] Master 127.7.170.62:36559 was elected leader, sending a full tablet report...
I20260929 02:18:55.378302 12430 ts_tablet_manager.cc:1428] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260929 02:18:55.378201 12431 consensus_queue.cc:260] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.379779 12431 ts_tablet_manager.cc:1428] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:18:55.381290 12432 tablet_bootstrap.cc:492] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: No bootstrap required, opened a new log
I20260929 02:18:55.381556 12432 ts_tablet_manager.cc:1397] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.006s
I20260929 02:18:55.388793 12432 raft_consensus.cc:348] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.389231 12432 raft_consensus.cc:374] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:55.389320 12432 raft_consensus.cc:729] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27045ff8e8424b838f0ae40eaa07a7a8, State: Initialized, Role: FOLLOWER
I20260929 02:18:55.389858 12432 consensus_queue.cc:260] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.390815 12412 heartbeater.cc:503] Master 127.7.170.62:36559 was elected leader, sending a full tablet report...
I20260929 02:18:55.391405 12432 ts_tablet_manager.cc:1428] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
W20260929 02:18:55.499598 12413 tablet.cc:2369] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:18:55.521674 12281 tablet.cc:2369] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:18:55.535087 12148 tablet.cc:2369] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:55.556913 12437 raft_consensus.cc:482] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:55.557211 12437 raft_consensus.cc:504] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.558858 12437 leader_election.cc:288] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea3f3df7c2274b139c84fce7737cbc56 (127.7.170.2:42271), 27045ff8e8424b838f0ae40eaa07a7a8 (127.7.170.3:42779)
I20260929 02:18:55.565142 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.559289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:36509 (local address 127.7.170.2:42271)
0929 02:18:55.559741 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.559747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.559938 (+ 191us) server_negotiation.cc:407] Connection header received
0929 02:18:55.560153 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.560160 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.560234 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.560319 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:55.560973 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.561700 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.563463 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.564204 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.564309 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.564401 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.564743 (+ 342us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.564863 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320}
I20260929 02:18:55.565146 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.559452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:41079 (local address 127.7.170.3:42779)
0929 02:18:55.559650 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.559655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.560205 (+ 550us) server_negotiation.cc:407] Connection header received
0929 02:18:55.560359 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.560365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.560438 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.560527 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:18:55.561130 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.561821 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.563488 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.564252 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.564370 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.564463 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.564780 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.564902 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:18:55.565241 12439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.559077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:42271 (local address 127.7.170.1:36509)
0929 02:18:55.559814 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.559836 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.560009 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.560343 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.560347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.560373 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:55.560804 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.560814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.561821 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.561826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.563320 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.563331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.564430 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.564436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.564590 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.564630 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.564696 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.564809 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":26,"thread_start_us":433,"threads_started":1}
I20260929 02:18:55.565526 12440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.559310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:42779 (local address 127.7.170.1:41079)
0929 02:18:55.560099 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:55.560115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:55.560234 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:55.560551 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:55.560554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:55.560576 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:18:55.560985 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.560997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.561951 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.561955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.563346 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:55.563358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.564475 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.564480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:55.564635 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.564658 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:18:55.564743 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 02:18:55.564861 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":58,"thread_start_us":471,"threads_started":1}
I20260929 02:18:55.566265 12235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d572f6650e4d9e8ab36b2dcb2628d9" candidate_uuid: "a356ffa348984ca6a337af19d9845b64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea3f3df7c2274b139c84fce7737cbc56" is_pre_election: true
I20260929 02:18:55.566573 12367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d572f6650e4d9e8ab36b2dcb2628d9" candidate_uuid: "a356ffa348984ca6a337af19d9845b64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" is_pre_election: true
I20260929 02:18:55.566778 12235 raft_consensus.cc:2454] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a356ffa348984ca6a337af19d9845b64 in term 0.
I20260929 02:18:55.567021 12367 raft_consensus.cc:2454] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a356ffa348984ca6a337af19d9845b64 in term 0.
I20260929 02:18:55.567415 12035 leader_election.cc:302] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a356ffa348984ca6a337af19d9845b64, ea3f3df7c2274b139c84fce7737cbc56; no voters:
I20260929 02:18:55.567832 12437 raft_consensus.cc:2781] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:55.567962 12437 raft_consensus.cc:482] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:55.568059 12437 raft_consensus.cc:3037] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:55.570199 12437 raft_consensus.cc:504] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.570705 12437 leader_election.cc:288] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [CANDIDATE]: Term 1 election: Requested vote from peers ea3f3df7c2274b139c84fce7737cbc56 (127.7.170.2:42271), 27045ff8e8424b838f0ae40eaa07a7a8 (127.7.170.3:42779)
I20260929 02:18:55.571089 12235 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d572f6650e4d9e8ab36b2dcb2628d9" candidate_uuid: "a356ffa348984ca6a337af19d9845b64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea3f3df7c2274b139c84fce7737cbc56"
I20260929 02:18:55.571095 12367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2d572f6650e4d9e8ab36b2dcb2628d9" candidate_uuid: "a356ffa348984ca6a337af19d9845b64" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27045ff8e8424b838f0ae40eaa07a7a8"
I20260929 02:18:55.571235 12367 raft_consensus.cc:3037] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:55.571247 12235 raft_consensus.cc:3037] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:55.573261 12367 raft_consensus.cc:2454] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a356ffa348984ca6a337af19d9845b64 in term 1.
I20260929 02:18:55.573347 12235 raft_consensus.cc:2454] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a356ffa348984ca6a337af19d9845b64 in term 1.
I20260929 02:18:55.573604 12038 leader_election.cc:302] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27045ff8e8424b838f0ae40eaa07a7a8, a356ffa348984ca6a337af19d9845b64; no voters:
I20260929 02:18:55.573865 12437 raft_consensus.cc:2781] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:55.574260 12437 raft_consensus.cc:686] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 1 LEADER]: Becoming Leader. State: Replica: a356ffa348984ca6a337af19d9845b64, State: Running, Role: LEADER
I20260929 02:18:55.574729 12437 consensus_queue.cc:237] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } }
I20260929 02:18:55.578743 11959 catalog_manager.cc:5495] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 reported cstate change: term changed from 0 to 1, leader changed from <none> to a356ffa348984ca6a337af19d9845b64 (127.7.170.1). New cstate: current_term: 1 leader_uuid: "a356ffa348984ca6a337af19d9845b64" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a356ffa348984ca6a337af19d9845b64" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38599 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 } health_report { overall_health: UNKNOWN } } }
I20260929 02:18:55.634879 12425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.630145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53010 (local address 127.7.170.1:38599)
0929 02:18:55.630343 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.630348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.630366 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:18:55.630514 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.630522 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.630599 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.630719 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:18:55.631036 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.631763 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.633045 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.633436 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.633533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.633928 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:18:55.634414 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:18:55.634537 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.634572 (+ 35us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.634634 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:18:55.642854 12235 raft_consensus.cc:1261] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 1 FOLLOWER]: Refusing update from remote peer a356ffa348984ca6a337af19d9845b64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:55.642854 12367 raft_consensus.cc:1261] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 1 FOLLOWER]: Refusing update from remote peer a356ffa348984ca6a337af19d9845b64: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:55.643878 12437 consensus_queue.cc:1035] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea3f3df7c2274b139c84fce7737cbc56" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 42271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:55.644188 12441 consensus_queue.cc:1035] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27045ff8e8424b838f0ae40eaa07a7a8" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 42779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:55.655071 12446 mvcc.cc:205] Tried to move back new op lower bound from 7334495582782914560 to 7334495582517542912. Current Snapshot: MvccSnapshot[applied={T|T < 7334495582782914560}]
I20260929 02:18:55.667133 11959 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:55.667362 11959 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:55.670883 11959 catalog_manager.cc:5782] T 00000000000000000000000000000000 P faffcd17ecdc486e8307503def79e8e8: Sending DeleteTablet for 3 replicas of tablet a2d572f6650e4d9e8ab36b2dcb2628d9
I20260929 02:18:55.672025 12082 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:55 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:53008
I20260929 02:18:55.672025 12347 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:55 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:49338
I20260929 02:18:55.672034 12215 tablet_service.cc:1514] Processing DeleteTablet for tablet a2d572f6650e4d9e8ab36b2dcb2628d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:55 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:46912
I20260929 02:18:55.674037 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12020
I20260929 02:18:55.689558 12455 tablet_replica.cc:330] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: stopping tablet replica
I20260929 02:18:55.690274 12455 raft_consensus.cc:2229] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:55.691126 12455 raft_consensus.cc:2258] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:55.693658 12455 ts_tablet_manager.cc:1905] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:55.695972 12456 tablet_replica.cc:330] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: stopping tablet replica
I20260929 02:18:55.696266 12454 tablet_replica.cc:330] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: stopping tablet replica
Sep 29 02:18:55 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:18:55.697314 12454 raft_consensus.cc:2229] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:55.697719 12456 raft_consensus.cc:2229] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:55.698302 12454 raft_consensus.cc:2258] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:55.698807 12456 raft_consensus.cc:2258] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:55.701555 12454 ts_tablet_manager.cc:1905] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:55.702081 12455 ts_tablet_manager.cc:1918] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:55.702281 12455 log.cc:1196] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/a2d572f6650e4d9e8ab36b2dcb2628d9
I20260929 02:18:55.703028 12455 ts_tablet_manager.cc:1939] T a2d572f6650e4d9e8ab36b2dcb2628d9 P a356ffa348984ca6a337af19d9845b64: Deleting consensus metadata
I20260929 02:18:55.703025 12456 ts_tablet_manager.cc:1905] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:55.704818 11944 catalog_manager.cc:4841] TS a356ffa348984ca6a337af19d9845b64 (127.7.170.1:38599): tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (table test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]) successfully deleted
I20260929 02:18:55.706772 12425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.677240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53026 (local address 127.7.170.1:38599)
0929 02:18:55.677434 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.677442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.680457 (+ 3015us) server_negotiation.cc:407] Connection header received
0929 02:18:55.681346 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.681352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.681451 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.681528 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:55.685654 (+ 4126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.686972 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.690759 (+ 3787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.691429 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.691529 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.698575 (+ 7046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:55.698589 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:55.698607 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:55.698657 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:55.703326 (+ 4669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.704178 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.704186 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.704189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.704376 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.704813 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.704817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.704821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.705276 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:55.705543 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.706133 (+ 590us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.706369 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:18:55.707492 12142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:55.711761 12454 ts_tablet_manager.cc:1918] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:55.711915 12454 log.cc:1196] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/a2d572f6650e4d9e8ab36b2dcb2628d9
I20260929 02:18:55.712309 12454 ts_tablet_manager.cc:1939] T a2d572f6650e4d9e8ab36b2dcb2628d9 P 27045ff8e8424b838f0ae40eaa07a7a8: Deleting consensus metadata
I20260929 02:18:55.713100 12456 ts_tablet_manager.cc:1918] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:55.713245 12456 log.cc:1196] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/a2d572f6650e4d9e8ab36b2dcb2628d9
I20260929 02:18:55.713723 12456 ts_tablet_manager.cc:1939] T a2d572f6650e4d9e8ab36b2dcb2628d9 P ea3f3df7c2274b139c84fce7737cbc56: Deleting consensus metadata
I20260929 02:18:55.713853 11946 catalog_manager.cc:4841] TS 27045ff8e8424b838f0ae40eaa07a7a8 (127.7.170.3:42779): tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (table test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]) successfully deleted
I20260929 02:18:55.715310 11943 catalog_manager.cc:4841] TS ea3f3df7c2274b139c84fce7737cbc56 (127.7.170.2:42271): tablet a2d572f6650e4d9e8ab36b2dcb2628d9 (table test-table [id=7560fa0f9a9e4d0ca9abfae9068627eb]) successfully deleted
I20260929 02:18:55.811772 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12020
I20260929 02:18:55.826813 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12153
Sep 29 02:18:55 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:18:55.841138 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.827557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46918 (local address 127.7.170.2:42271)
0929 02:18:55.827727 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.827732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.827760 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:18:55.827954 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.827960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.828049 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.828143 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:55.829349 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.830063 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.831253 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.831635 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.831738 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.835533 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:55.835547 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:55.835563 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:55.835598 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:55.838292 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.838945 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.838954 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.838956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.839094 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.839430 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.839433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.839435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.839873 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:55.840222 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.840542 (+ 320us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.840754 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:18:55.841984 12275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:55.930757 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12153
I20260929 02:18:55.943765 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12285
Sep 29 02:18:55 dist-test-slave-8cs2 krb5kdc[11915](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:18:55.957659 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:55.944503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49340 (local address 127.7.170.3:42779)
0929 02:18:55.944669 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:55.944674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:55.944693 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:18:55.944870 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:55.944874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:55.944983 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:55.945070 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:55.946257 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.946952 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.948161 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:55.948527 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:55.948632 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:55.952475 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:55.952490 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:55.952505 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:18:55.952542 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:55.955011 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.955597 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.955605 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.955607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.955737 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:18:55.956093 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:18:55.956098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:18:55.956101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:18:55.956550 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:55.956831 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:55.957116 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:18:55.957323 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:18:55.958303 12407 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:56.043349 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12285
I20260929 02:18:56.056154 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 11928
I20260929 02:18:56.057231 11989 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:56.164656 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 11928
2026-09-29T02:18:56Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4006 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-29T02:18:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:56Z Disabled control of system clock
I20260929 02:18:56.224438 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40613
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:18:56.526665 12465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:56.526954 12465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:56.527034 12465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:56.534188 12465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:56.534307 12465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:56.534343 12465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:56.534390 12465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:56.534432 12465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:18:56.543527 12465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:40613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:56.544739 12465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:56.546515 12465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:56.557588 12470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:56.557658 12471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:56.557621 12473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:56.558561 12465 server_base.cc:1034] running on GCE node
I20260929 02:18:56.559347 12465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:56.560721 12465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:56.561995 12465 hybrid_clock.cc:648] HybridClock initialized: now 1790648336561910 us; error 47 us; skew 500 ppm
I20260929 02:18:56.562409 12465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:56.564500 12465 webserver.cc:460] Webserver started at http://127.7.170.62:35693/ using document root <none> and password file <none>
I20260929 02:18:56.565207 12465 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:56.565320 12465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:56.565613 12465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:56.568493 12465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a"
format_stamp: "Formatted at 2026-09-29 02:18:56 on dist-test-slave-8cs2"
server_key: "9a3ca7c73ab5b969d7e7c73a8d8f2cd2"
server_key_iv: "c78c8865567cc67be848bb9b25c9345f"
server_key_version: "encryptionkey@0"
I20260929 02:18:56.569324 12465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a"
format_stamp: "Formatted at 2026-09-29 02:18:56 on dist-test-slave-8cs2"
server_key: "9a3ca7c73ab5b969d7e7c73a8d8f2cd2"
server_key_iv: "c78c8865567cc67be848bb9b25c9345f"
server_key_version: "encryptionkey@0"
I20260929 02:18:56.574353 12465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 02:18:56.577694 12479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:56.579008 12465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:56.579164 12465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a"
format_stamp: "Formatted at 2026-09-29 02:18:56 on dist-test-slave-8cs2"
server_key: "9a3ca7c73ab5b969d7e7c73a8d8f2cd2"
server_key_iv: "c78c8865567cc67be848bb9b25c9345f"
server_key_version: "encryptionkey@0"
I20260929 02:18:56.579351 12465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:56.585067 12465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:56.585813 12465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:56.586079 12465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:56.602602 12465 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:40613
I20260929 02:18:56.602635 12531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:40613 every 8 connection(s)
I20260929 02:18:56.603855 12465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:56.607491 12532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:56.610507 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12465
I20260929 02:18:56.610671 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:56.611238 7848 external_mini_cluster.cc:1428] Setting key b0168ded109f9343fdcded10a7a506f8
I20260929 02:18:56.618716 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:56.613069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33168 (local address 127.7.170.62:40613)
0929 02:18:56.613946 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:56.613956 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:56.614002 (+ 46us) server_negotiation.cc:407] Connection header received
0929 02:18:56.615448 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:56.615469 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:56.615897 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:56.616225 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:56.617237 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:56.617243 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:56.617255 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:56.617268 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:56.617331 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:56.617346 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:56.617502 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:56.617613 (+ 111us) server_negotiation.cc:299] Negotiation successful
0929 02:18:56.617836 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":304,"threads_started":1}
I20260929 02:18:56.619270 12532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: Bootstrap starting.
I20260929 02:18:56.622383 12532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:56.623265 12532 log.cc:824] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:56.625699 12532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: No bootstrap required, opened a new log
I20260929 02:18:56.632899 12532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } }
I20260929 02:18:56.633232 12532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:56.633320 12532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a6a6b10fbd64e1d950adae1d9cb2e4a, State: Initialized, Role: FOLLOWER
I20260929 02:18:56.633867 12532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } }
I20260929 02:18:56.634075 12532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:56.634194 12532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:56.634325 12532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:56.636273 12532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } }
I20260929 02:18:56.636700 12532 leader_election.cc:302] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a6a6b10fbd64e1d950adae1d9cb2e4a; no voters:
I20260929 02:18:56.637316 12532 leader_election.cc:288] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:56.637425 12537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:56.638055 12537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [term 1 LEADER]: Becoming Leader. State: Replica: 8a6a6b10fbd64e1d950adae1d9cb2e4a, State: Running, Role: LEADER
I20260929 02:18:56.638558 12537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } }
I20260929 02:18:56.639181 12532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:56.641177 12538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } } }
I20260929 02:18:56.641106 12539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a6a6b10fbd64e1d950adae1d9cb2e4a. Latest consensus state: current_term: 1 leader_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a6a6b10fbd64e1d950adae1d9cb2e4a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40613 } } }
I20260929 02:18:56.641495 12538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:56.641499 12539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:56.642809 12544 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:56.647893 12544 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:56.655609 12544 catalog_manager.cc:1347] Generated new cluster ID: 2b78e235e28d4fb6802e732217419cb7
I20260929 02:18:56.655709 12544 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:56.677280 12544 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:56.677666 12544 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:56.685806 12544 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: Generated new TSK 0
I20260929 02:18:56.686812 12544 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:18:56.691888 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--builtin_ntp_servers=127.7.170.20:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260929 02:18:57.005062 12556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:57.005395 12556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:57.005527 12556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:57.012905 12556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:57.013049 12556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:57.013204 12556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:18:57.022476 12556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:57.023763 12556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:57.025568 12556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:57.037838 12561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.037884 12564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.037909 12562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:57.038594 12556 server_base.cc:1034] running on GCE node
I20260929 02:18:57.039144 12556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:57.040097 12556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:57.041311 12556 hybrid_clock.cc:648] HybridClock initialized: now 1790648337041262 us; error 41 us; skew 500 ppm
I20260929 02:18:57.041617 12556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:57.043725 12556 webserver.cc:460] Webserver started at http://127.7.170.1:35807/ using document root <none> and password file <none>
I20260929 02:18:57.044354 12556 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:57.044458 12556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:57.044737 12556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:57.047462 12556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "7df8cb4a09b448a0828e9842adae61c8"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "602947fcef9ce7e72eba32a602e27b86"
server_key_iv: "ff207c3090ec772bb1b00c2c09fbf323"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.048174 12556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "7df8cb4a09b448a0828e9842adae61c8"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "602947fcef9ce7e72eba32a602e27b86"
server_key_iv: "ff207c3090ec772bb1b00c2c09fbf323"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.052865 12556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:18:57.055962 12570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.057209 12556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260929 02:18:57.057336 12556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "7df8cb4a09b448a0828e9842adae61c8"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "602947fcef9ce7e72eba32a602e27b86"
server_key_iv: "ff207c3090ec772bb1b00c2c09fbf323"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.057539 12556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:57.062605 12556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:57.063289 12556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:57.063529 12556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:57.064316 12556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:57.065577 12556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:57.065665 12556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.065763 12556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:57.065845 12556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.094321 12556 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:44835
I20260929 02:18:57.094367 12683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:44835 every 8 connection(s)
I20260929 02:18:57.095574 12556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:18:57.101280 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12556
I20260929 02:18:57.101465 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:18:57.102104 7848 external_mini_cluster.cc:1428] Setting key 4a036dd6c5b6cdcd0490188c28c851ac
I20260929 02:18:57.103273 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.098026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:59835 (local address 127.7.170.62:40613)
0929 02:18:57.098186 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.098190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.099510 (+ 1320us) server_negotiation.cc:407] Connection header received
0929 02:18:57.101136 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.101143 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.101210 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.101323 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.102575 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.102581 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.102585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.102602 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.102640 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.102654 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.102743 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.102959 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.103019 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:18:57.104206 12686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.098518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40613 (local address 127.7.170.1:59835)
0929 02:18:57.099359 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.099406 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.100853 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.101528 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.101539 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.101643 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.102290 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.102293 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.102328 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.102418 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.102739 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.102748 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.102756 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.103012 (+ 256us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.103237 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":267,"threads_started":1}
I20260929 02:18:57.104624 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--builtin_ntp_servers=127.7.170.20:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 02:18:57.105530 12684 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40613
I20260929 02:18:57.105835 12684 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:57.106468 12684 heartbeater.cc:511] Master 127.7.170.62:40613 requested a full tablet report, sending...
I20260929 02:18:57.108415 12496 ts_manager.cc:194] Registered new tserver with Master: 7df8cb4a09b448a0828e9842adae61c8 (127.7.170.1:44835)
W20260929 02:18:57.411345 12687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:57.411600 12687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:57.411684 12687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:57.418828 12687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:57.418922 12687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:57.419040 12687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:18:57.427665 12687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:57.428808 12687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:57.430545 12687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:57.442680 12693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.442704 12695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.443239 12692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:57.443430 12687 server_base.cc:1034] running on GCE node
I20260929 02:18:57.444054 12687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:57.445017 12687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:57.446209 12687 hybrid_clock.cc:648] HybridClock initialized: now 1790648337446153 us; error 46 us; skew 500 ppm
I20260929 02:18:57.446496 12687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:57.448596 12687 webserver.cc:460] Webserver started at http://127.7.170.2:41003/ using document root <none> and password file <none>
I20260929 02:18:57.449317 12687 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:57.449415 12687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:57.449704 12687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:57.452448 12687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "13032d2236754fa1899f6e56530e341c"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "88d30d74a9d991558f7d89194e266b9f"
server_key_iv: "dfc3896fb229745877a88e878159abc2"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.453225 12687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "13032d2236754fa1899f6e56530e341c"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "88d30d74a9d991558f7d89194e266b9f"
server_key_iv: "dfc3896fb229745877a88e878159abc2"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.457945 12687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:18:57.461165 12701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.462451 12687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:18:57.462607 12687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "13032d2236754fa1899f6e56530e341c"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "88d30d74a9d991558f7d89194e266b9f"
server_key_iv: "dfc3896fb229745877a88e878159abc2"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.462807 12687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:57.468055 12687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:57.468789 12687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:57.469061 12687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:57.469923 12687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:57.471283 12687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:57.471386 12687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.471488 12687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:57.471557 12687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.499692 12687 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44545
I20260929 02:18:57.499783 12814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44545 every 8 connection(s)
I20260929 02:18:57.500918 12687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:18:57.503537 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12687
I20260929 02:18:57.503688 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:18:57.504195 7848 external_mini_cluster.cc:1428] Setting key a2f9275e83f3bb7fa557a333640c41b5
I20260929 02:18:57.506732 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--builtin_ntp_servers=127.7.170.20:42545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260929 02:18:57.509022 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.503293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:43501 (local address 127.7.170.62:40613)
0929 02:18:57.503456 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.503461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.505135 (+ 1674us) server_negotiation.cc:407] Connection header received
0929 02:18:57.506837 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.506842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.506902 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.506983 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.508290 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.508297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.508301 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.508335 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.508366 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.508377 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.508475 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.508655 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.508716 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:18:57.509929 12817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.503944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40613 (local address 127.7.170.2:43501)
0929 02:18:57.504985 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.505037 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.506599 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.507197 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.507205 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.507294 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.507994 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.507997 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.508035 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.508118 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.508462 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.508472 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.508479 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.508720 (+ 241us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.509035 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":324,"threads_started":1}
I20260929 02:18:57.511198 12815 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40613
I20260929 02:18:57.511466 12815 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:57.512033 12815 heartbeater.cc:511] Master 127.7.170.62:40613 requested a full tablet report, sending...
I20260929 02:18:57.513366 12496 ts_manager.cc:194] Registered new tserver with Master: 13032d2236754fa1899f6e56530e341c (127.7.170.2:44545)
W20260929 02:18:57.815193 12818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:57.815487 12818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:57.815620 12818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:57.822860 12818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:57.822963 12818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:57.823107 12818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:18:57.832028 12818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:57.833343 12818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:57.835047 12818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:57.847098 12823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.847064 12824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:57.847136 12826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:57.847996 12818 server_base.cc:1034] running on GCE node
I20260929 02:18:57.848582 12818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:57.849491 12818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:57.850695 12818 hybrid_clock.cc:648] HybridClock initialized: now 1790648337850641 us; error 47 us; skew 500 ppm
I20260929 02:18:57.851044 12818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:57.853152 12818 webserver.cc:460] Webserver started at http://127.7.170.3:37627/ using document root <none> and password file <none>
I20260929 02:18:57.853792 12818 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:57.853892 12818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:57.854175 12818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:57.856812 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "676aee54473e435580213ef1546c4e0d"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "efa0f4b7a3b633a5ed58bd3e1fd5ad10"
server_key_iv: "8002ea83573b7d3c22019b24b4b1a489"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.857579 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "676aee54473e435580213ef1546c4e0d"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "efa0f4b7a3b633a5ed58bd3e1fd5ad10"
server_key_iv: "8002ea83573b7d3c22019b24b4b1a489"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.862349 12818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:18:57.865525 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.866801 12818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:57.866966 12818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "676aee54473e435580213ef1546c4e0d"
format_stamp: "Formatted at 2026-09-29 02:18:57 on dist-test-slave-8cs2"
server_key: "efa0f4b7a3b633a5ed58bd3e1fd5ad10"
server_key_iv: "8002ea83573b7d3c22019b24b4b1a489"
server_key_version: "encryptionkey@0"
I20260929 02:18:57.867182 12818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:57.872419 12818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:57.873188 12818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:57.873422 12818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:57.874267 12818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:18:57.875526 12818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:18:57.875614 12818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.875715 12818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:18:57.875784 12818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:57.903781 12818 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:41929
I20260929 02:18:57.903826 12946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:41929 every 8 connection(s)
I20260929 02:18:57.905088 12818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:18:57.912465 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.907440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:42791 (local address 127.7.170.62:40613)
0929 02:18:57.907611 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.907616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.908976 (+ 1360us) server_negotiation.cc:407] Connection header received
0929 02:18:57.910626 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.910630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.910683 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.910765 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.911934 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.911938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.911941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.911955 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.911988 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.912000 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.912096 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.912234 (+ 138us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.912279 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:18:57.913420 12949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.907948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40613 (local address 127.7.170.3:42791)
0929 02:18:57.908801 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.908851 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.910404 (+ 1553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.910981 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.910988 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.911078 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.911658 (+ 580us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.911660 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.911697 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.911783 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.912071 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.912076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.912081 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.912296 (+ 215us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.912489 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":251,"threads_started":1}
I20260929 02:18:57.914278 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 12818
I20260929 02:18:57.914444 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:18:57.914736 12947 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40613
I20260929 02:18:57.914944 7848 external_mini_cluster.cc:1428] Setting key c58ade9d899c198fc772971435ff873a
I20260929 02:18:57.915017 12947 heartbeater.cc:464] Registering TS with master...
I20260929 02:18:57.915635 12947 heartbeater.cc:511] Master 127.7.170.62:40613 requested a full tablet report, sending...
I20260929 02:18:57.917225 12495 ts_manager.cc:194] Registered new tserver with Master: 676aee54473e435580213ef1546c4e0d (127.7.170.3:41929)
I20260929 02:18:57.929834 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:18:57.936025 12535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.933918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.7.170.62:40613)
0929 02:18:57.934124 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.934132 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.934516 (+ 384us) server_negotiation.cc:407] Connection header received
0929 02:18:57.934648 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.934652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.934703 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.934779 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.935515 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.935519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.935522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.935535 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.935563 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.935574 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.935662 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.935710 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.935749 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:18:57.939610 12495 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:18:57.946237 12495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:18:57.964258 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.957677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.7.170.3:41929)
0929 02:18:57.958754 (+ 1077us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.958764 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.960155 (+ 1391us) server_negotiation.cc:407] Connection header received
0929 02:18:57.960355 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.960363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.960611 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.960746 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.963325 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.963332 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.963341 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.963360 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.963431 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.963448 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.963581 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.963809 (+ 228us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.963898 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":722,"threads_started":1}
I20260929 02:18:57.964289 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.957251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44545 (local address 127.0.0.1:34878)
0929 02:18:57.960468 (+ 3217us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.960483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.960566 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.962087 (+ 1521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.962095 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.962183 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.963121 (+ 938us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.963124 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.963135 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.963163 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.963626 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.963636 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.963644 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.963785 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.963866 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3094,"thread_start_us":2422,"threads_started":1}
I20260929 02:18:57.965121 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.958328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.7.170.2:44545)
0929 02:18:57.959527 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.959536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.960664 (+ 1128us) server_negotiation.cc:407] Connection header received
0929 02:18:57.960729 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.960737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.960979 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.961112 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.963318 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.963326 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.963341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.963364 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.963414 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.963434 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.963576 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.964679 (+ 1103us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.964759 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1038,"thread_start_us":884,"threads_started":1}
I20260929 02:18:57.966544 12959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.957194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:41929 (local address 127.0.0.1:40184)
0929 02:18:57.960037 (+ 2843us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.960071 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.960191 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.962087 (+ 1896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.962095 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.962183 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.963046 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.963049 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.963082 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.963155 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.963654 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.963658 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.963662 (+ 4us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.965195 (+ 1533us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.966082 (+ 887us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2651,"thread_start_us":1725,"threads_started":1}
I20260929 02:18:57.968292 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.959253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44835 (local address 127.0.0.1:38382)
0929 02:18:57.964456 (+ 5203us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:57.964474 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:57.964545 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:57.966219 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:57.966224 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:57.966271 (+ 47us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:57.966839 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:57.966842 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:57.966857 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:57.966907 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:57.967896 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:57.967901 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:57.967907 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:18:57.968025 (+ 118us) client_negotiation.cc:240] Negotiation successful
0929 02:18:57.968070 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5126,"thread_start_us":6627,"threads_started":1}
I20260929 02:18:57.968405 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:57.959859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38382 (local address 127.7.170.1:44835)
0929 02:18:57.965722 (+ 5863us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:57.965730 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:57.965756 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:18:57.965829 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:57.965836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:57.966039 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:57.966593 (+ 554us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:57.967688 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:57.967695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:57.967706 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:57.967720 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:57.967767 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:57.967789 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:57.967926 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:57.968045 (+ 119us) server_negotiation.cc:299] Negotiation successful
0929 02:18:57.968134 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5719,"thread_start_us":5615,"threads_started":1}
I20260929 02:18:57.969229 12750 tablet_service.cc:1467] Processing CreateTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 (DEFAULT_TABLE table=test-table [id=3a0de7e76792408b80a62b580c92f796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:57.970618 12750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675bdfc86c68453ba25e2466b79fb8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:57.970243 12882 tablet_service.cc:1467] Processing CreateTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 (DEFAULT_TABLE table=test-table [id=3a0de7e76792408b80a62b580c92f796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:57.971524 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675bdfc86c68453ba25e2466b79fb8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:57.971850 12619 tablet_service.cc:1467] Processing CreateTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 (DEFAULT_TABLE table=test-table [id=3a0de7e76792408b80a62b580c92f796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:18:57.973129 12619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 675bdfc86c68453ba25e2466b79fb8a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:57.982175 12965 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Bootstrap starting.
I20260929 02:18:57.985677 12966 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Bootstrap starting.
I20260929 02:18:57.986166 12965 tablet_bootstrap.cc:654] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:57.986871 12967 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Bootstrap starting.
I20260929 02:18:57.987394 12965 log.cc:824] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:57.988888 12966 tablet_bootstrap.cc:654] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:57.989880 12967 tablet_bootstrap.cc:654] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:57.990083 12966 log.cc:824] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:57.990561 12965 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: No bootstrap required, opened a new log
I20260929 02:18:57.990831 12965 ts_tablet_manager.cc:1397] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:18:57.990873 12967 log.cc:824] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:57.993043 12966 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: No bootstrap required, opened a new log
I20260929 02:18:57.993286 12966 ts_tablet_manager.cc:1397] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:18:57.999622 12967 tablet_bootstrap.cc:492] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: No bootstrap required, opened a new log
I20260929 02:18:57.999861 12967 ts_tablet_manager.cc:1397] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260929 02:18:58.000562 12965 raft_consensus.cc:348] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.001021 12965 raft_consensus.cc:374] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:58.001122 12965 raft_consensus.cc:729] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676aee54473e435580213ef1546c4e0d, State: Initialized, Role: FOLLOWER
I20260929 02:18:58.001731 12965 consensus_queue.cc:260] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.003186 12966 raft_consensus.cc:348] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.003629 12966 raft_consensus.cc:374] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:58.003726 12966 raft_consensus.cc:729] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13032d2236754fa1899f6e56530e341c, State: Initialized, Role: FOLLOWER
I20260929 02:18:58.004361 12966 consensus_queue.cc:260] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.006189 12966 ts_tablet_manager.cc:1428] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:18:58.006796 12815 heartbeater.cc:503] Master 127.7.170.62:40613 was elected leader, sending a full tablet report...
I20260929 02:18:58.007634 12967 raft_consensus.cc:348] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.007949 12967 raft_consensus.cc:374] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:58.008059 12967 raft_consensus.cc:729] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df8cb4a09b448a0828e9842adae61c8, State: Initialized, Role: FOLLOWER
I20260929 02:18:58.008634 12967 consensus_queue.cc:260] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.009618 12947 heartbeater.cc:503] Master 127.7.170.62:40613 was elected leader, sending a full tablet report...
I20260929 02:18:58.010298 12965 ts_tablet_manager.cc:1428] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Time spent starting tablet: real 0.019s user 0.008s sys 0.011s
I20260929 02:18:58.011247 12684 heartbeater.cc:503] Master 127.7.170.62:40613 was elected leader, sending a full tablet report...
I20260929 02:18:58.011973 12967 ts_tablet_manager.cc:1428] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260929 02:18:58.087208 12973 raft_consensus.cc:482] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:18:58.087512 12973 raft_consensus.cc:504] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.089473 12973 leader_election.cc:288] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 676aee54473e435580213ef1546c4e0d (127.7.170.3:41929), 13032d2236754fa1899f6e56530e341c (127.7.170.2:44545)
I20260929 02:18:58.092572 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.089980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:48613 (local address 127.7.170.3:41929)
0929 02:18:58.090140 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.090146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.090756 (+ 610us) server_negotiation.cc:407] Connection header received
0929 02:18:58.090927 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.090932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.090980 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.091068 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.091997 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.092004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.092008 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.092028 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.092067 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.092081 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.092210 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.092255 (+ 45us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.092312 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:18:58.092612 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.089904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:56565 (local address 127.7.170.2:44545)
0929 02:18:58.090085 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.090091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.091046 (+ 955us) server_negotiation.cc:407] Connection header received
0929 02:18:58.091162 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.091167 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.091221 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.091292 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.092087 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.092092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.092096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.092114 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.092149 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.092163 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.092284 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.092351 (+ 67us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.092401 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:18:58.092829 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.090023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44545 (local address 127.7.170.1:56565)
0929 02:18:58.090949 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:58.090963 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:58.091058 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:58.091347 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:58.091352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:58.091371 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:58.091907 (+ 536us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:58.091910 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:58.091923 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:58.091956 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:58.092257 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:58.092261 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:58.092269 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:18:58.092392 (+ 123us) client_negotiation.cc:240] Negotiation successful
0929 02:18:58.092427 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":613,"threads_started":1}
I20260929 02:18:58.092952 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.089814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:41929 (local address 127.7.170.1:48613)
0929 02:18:58.090650 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:58.090665 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:58.090807 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:58.091116 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:58.091120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:58.091143 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:18:58.091785 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:18:58.091789 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:18:58.091808 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:18:58.091860 (+ 52us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:18:58.092168 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:18:58.092171 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:18:58.092174 (+ 3us) client_negotiation.cc:769] Sending connection context
0929 02:18:58.092289 (+ 115us) client_negotiation.cc:240] Negotiation successful
0929 02:18:58.092341 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":447,"threads_started":1}
I20260929 02:18:58.093714 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675bdfc86c68453ba25e2466b79fb8a8" candidate_uuid: "7df8cb4a09b448a0828e9842adae61c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13032d2236754fa1899f6e56530e341c" is_pre_election: true
I20260929 02:18:58.094002 12902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675bdfc86c68453ba25e2466b79fb8a8" candidate_uuid: "7df8cb4a09b448a0828e9842adae61c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676aee54473e435580213ef1546c4e0d" is_pre_election: true
I20260929 02:18:58.094153 12770 raft_consensus.cc:2454] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df8cb4a09b448a0828e9842adae61c8 in term 0.
I20260929 02:18:58.094483 12902 raft_consensus.cc:2454] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7df8cb4a09b448a0828e9842adae61c8 in term 0.
I20260929 02:18:58.094718 12572 leader_election.cc:302] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13032d2236754fa1899f6e56530e341c, 7df8cb4a09b448a0828e9842adae61c8; no voters:
I20260929 02:18:58.095077 12973 raft_consensus.cc:2781] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:18:58.095223 12973 raft_consensus.cc:482] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:18:58.095326 12973 raft_consensus.cc:3037] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:18:58.097488 12685 tablet.cc:2369] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:18:58.097640 12973 raft_consensus.cc:504] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.098315 12973 leader_election.cc:288] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [CANDIDATE]: Term 1 election: Requested vote from peers 676aee54473e435580213ef1546c4e0d (127.7.170.3:41929), 13032d2236754fa1899f6e56530e341c (127.7.170.2:44545)
I20260929 02:18:58.098623 12902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675bdfc86c68453ba25e2466b79fb8a8" candidate_uuid: "7df8cb4a09b448a0828e9842adae61c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676aee54473e435580213ef1546c4e0d"
I20260929 02:18:58.098812 12902 raft_consensus.cc:3037] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:58.098953 12770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "675bdfc86c68453ba25e2466b79fb8a8" candidate_uuid: "7df8cb4a09b448a0828e9842adae61c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13032d2236754fa1899f6e56530e341c"
I20260929 02:18:58.099148 12770 raft_consensus.cc:3037] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:58.101910 12902 raft_consensus.cc:2454] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df8cb4a09b448a0828e9842adae61c8 in term 1.
I20260929 02:18:58.102137 12770 raft_consensus.cc:2454] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7df8cb4a09b448a0828e9842adae61c8 in term 1.
I20260929 02:18:58.102411 12574 leader_election.cc:302] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676aee54473e435580213ef1546c4e0d, 7df8cb4a09b448a0828e9842adae61c8; no voters:
I20260929 02:18:58.102699 12973 raft_consensus.cc:2781] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:58.103164 12973 raft_consensus.cc:686] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 1 LEADER]: Becoming Leader. State: Replica: 7df8cb4a09b448a0828e9842adae61c8, State: Running, Role: LEADER
I20260929 02:18:58.103729 12973 consensus_queue.cc:237] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } }
I20260929 02:18:58.107929 12496 catalog_manager.cc:5495] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7df8cb4a09b448a0828e9842adae61c8 (127.7.170.1). New cstate: current_term: 1 leader_uuid: "7df8cb4a09b448a0828e9842adae61c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7df8cb4a09b448a0828e9842adae61c8" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44835 } health_report { overall_health: HEALTHY } } }
I20260929 02:18:58.127033 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.125193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.7.170.1:44835)
0929 02:18:58.125359 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.125365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.125487 (+ 122us) server_negotiation.cc:407] Connection header received
0929 02:18:58.125662 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.125665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.125739 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.125818 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.126523 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.126526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.126530 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.126542 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.126571 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.126582 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.126659 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.126715 (+ 56us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.126754 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:18:58.134413 12770 raft_consensus.cc:1261] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 1 FOLLOWER]: Refusing update from remote peer 7df8cb4a09b448a0828e9842adae61c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:58.134430 12902 raft_consensus.cc:1261] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 1 FOLLOWER]: Refusing update from remote peer 7df8cb4a09b448a0828e9842adae61c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:18:58.135268 12976 consensus_queue.cc:1035] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676aee54473e435580213ef1546c4e0d" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 41929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:58.135555 12973 consensus_queue.cc:1035] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13032d2236754fa1899f6e56530e341c" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:18:58.144436 12979 mvcc.cc:205] Tried to move back new op lower bound from 7334495592990216192 to 7334495592876437504. Current Snapshot: MvccSnapshot[applied={T|T < 7334495592990216192}]
I20260929 02:18:58.145923 12980 mvcc.cc:205] Tried to move back new op lower bound from 7334495592990216192 to 7334495592876437504. Current Snapshot: MvccSnapshot[applied={T|T < 7334495592990216192}]
I20260929 02:18:58.151293 12982 mvcc.cc:205] Tried to move back new op lower bound from 7334495592990216192 to 7334495592876437504. Current Snapshot: MvccSnapshot[applied={T|T < 7334495592990216192}]
I20260929 02:18:58.158609 12495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:58.158800 12495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:18:58.162004 12495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a6a6b10fbd64e1d950adae1d9cb2e4a: Sending DeleteTablet for 3 replicas of tablet 675bdfc86c68453ba25e2466b79fb8a8
I20260929 02:18:58.163087 12882 tablet_service.cc:1514] Processing DeleteTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:58 UTC) from {username='slave'} at 127.0.0.1:40184
I20260929 02:18:58.163092 12750 tablet_service.cc:1514] Processing DeleteTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:58 UTC) from {username='slave'} at 127.0.0.1:34878
I20260929 02:18:58.163341 12619 tablet_service.cc:1514] Processing DeleteTablet for tablet 675bdfc86c68453ba25e2466b79fb8a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:18:58 UTC) from {username='slave'} at 127.0.0.1:38382
I20260929 02:18:58.165005 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12556
I20260929 02:18:58.175429 12991 tablet_replica.cc:330] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: stopping tablet replica
I20260929 02:18:58.175559 12989 tablet_replica.cc:330] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: stopping tablet replica
I20260929 02:18:58.175863 12991 raft_consensus.cc:2229] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:18:58.175967 12989 raft_consensus.cc:2229] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:58.176429 12989 raft_consensus.cc:2258] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:58.176499 12991 raft_consensus.cc:2258] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:58.178473 12989 ts_tablet_manager.cc:1905] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:58.178560 12991 ts_tablet_manager.cc:1905] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:58.185156 12989 ts_tablet_manager.cc:1918] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:58.185312 12989 log.cc:1196] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/675bdfc86c68453ba25e2466b79fb8a8
I20260929 02:18:58.185777 12991 ts_tablet_manager.cc:1918] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:58.185812 12989 ts_tablet_manager.cc:1939] T 675bdfc86c68453ba25e2466b79fb8a8 P 676aee54473e435580213ef1546c4e0d: Deleting consensus metadata
I20260929 02:18:58.185947 12991 log.cc:1196] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/675bdfc86c68453ba25e2466b79fb8a8
I20260929 02:18:58.186444 12991 ts_tablet_manager.cc:1939] T 675bdfc86c68453ba25e2466b79fb8a8 P 7df8cb4a09b448a0828e9842adae61c8: Deleting consensus metadata
I20260929 02:18:58.187587 12483 catalog_manager.cc:4841] TS 676aee54473e435580213ef1546c4e0d (127.7.170.3:41929): tablet 675bdfc86c68453ba25e2466b79fb8a8 (table test-table [id=3a0de7e76792408b80a62b580c92f796]) successfully deleted
I20260929 02:18:58.187935 12480 catalog_manager.cc:4841] TS 7df8cb4a09b448a0828e9842adae61c8 (127.7.170.1:44835): tablet 675bdfc86c68453ba25e2466b79fb8a8 (table test-table [id=3a0de7e76792408b80a62b580c92f796]) successfully deleted
I20260929 02:18:58.188285 12990 tablet_replica.cc:330] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: stopping tablet replica
I20260929 02:18:58.188697 12990 raft_consensus.cc:2229] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:18:58.189218 12990 raft_consensus.cc:2258] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:18:58.190131 12964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.166470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.7.170.1:44835)
0929 02:18:58.166629 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.166636 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.187790 (+ 21154us) server_negotiation.cc:407] Connection header received
0929 02:18:58.188360 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.188365 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.188423 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.188519 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.189412 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.189418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.189422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.189443 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.189485 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.189497 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.189615 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.189801 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.189859 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 02:18:58.190842 12679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:58.191413 12990 ts_tablet_manager.cc:1905] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:18:58.197999 12990 ts_tablet_manager.cc:1918] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:18:58.198143 12990 log.cc:1196] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/675bdfc86c68453ba25e2466b79fb8a8
I20260929 02:18:58.198593 12990 ts_tablet_manager.cc:1939] T 675bdfc86c68453ba25e2466b79fb8a8 P 13032d2236754fa1899f6e56530e341c: Deleting consensus metadata
I20260929 02:18:58.200201 12481 catalog_manager.cc:4841] TS 13032d2236754fa1899f6e56530e341c (127.7.170.2:44545): tablet 675bdfc86c68453ba25e2466b79fb8a8 (table test-table [id=3a0de7e76792408b80a62b580c92f796]) successfully deleted
I20260929 02:18:58.283834 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12556
I20260929 02:18:58.298444 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12687
I20260929 02:18:58.301112 12962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.299152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.7.170.2:44545)
0929 02:18:58.299316 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.299321 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.299341 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:18:58.299518 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.299522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.299608 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.299698 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.300512 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.300517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.300521 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.300535 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.300566 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.300577 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.300685 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.300766 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.300824 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:18:58.301666 12810 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:58.387027 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12687
I20260929 02:18:58.400549 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12818
I20260929 02:18:58.403285 12961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:58.401298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40198 (local address 127.7.170.3:41929)
0929 02:18:58.401459 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:58.401463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:58.401481 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:18:58.401608 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:58.401611 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:58.401687 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:58.401778 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:58.402709 (+ 931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:58.402714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:58.402717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:58.402733 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:58.402766 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:58.402778 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:58.402925 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:58.403018 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 02:18:58.403065 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:18:58.403944 12942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:58.488528 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12818
I20260929 02:18:58.501351 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 12465
I20260929 02:18:58.502395 12527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:58.602447 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 12465
2026-09-29T02:18:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2436 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260929 02:18:58.638244 7848 mini_oidc.cc:152] Starting JWKS server
I20260929 02:18:58.640659 7848 webserver.cc:460] Webserver started at https://127.0.0.1:44791/ using document root <none> and password file <none>
I20260929 02:18:58.640959 7848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:18:58.641541 7848 webserver.cc:460] Webserver started at http://127.0.0.1:39765/ using document root <none> and password file <none>
2026-09-29T02:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:58Z Disabled control of system clock
I20260929 02:18:58.665810 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44709
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:41405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:44709
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/testchainca.cert with env {}
W20260929 02:18:58.971390 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:58.971669 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:58.971760 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:58.978798 13002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:58.978899 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:58.978933 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:58.978992 13002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:58.979034 13002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:18:58.979092 13002 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:18:58.979130 13002 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44791/jwks.json
I20260929 02:18:58.988840 13002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:44709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44791/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:58.990126 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:58.991325 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:59.002338 13008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:59.002413 13007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:59.002413 13010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:59.003497 13002 server_base.cc:1034] running on GCE node
I20260929 02:18:59.004246 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:59.005607 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:59.006795 13002 hybrid_clock.cc:648] HybridClock initialized: now 1790648339006746 us; error 47 us; skew 500 ppm
I20260929 02:18:59.007119 13002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:59.013278 13002 webserver.cc:460] Webserver started at http://127.7.170.62:37869/ using document root <none> and password file <none>
I20260929 02:18:59.013971 13002 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:59.014073 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:59.014348 13002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:59.017199 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "659dea626aad4321a96c89817ba2c751"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "e352b206d25245ccad056334d643b34c"
server_key_iv: "7563bb57fa94d45f1182470afc8ba3e0"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.017944 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "659dea626aad4321a96c89817ba2c751"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "e352b206d25245ccad056334d643b34c"
server_key_iv: "7563bb57fa94d45f1182470afc8ba3e0"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.022819 13002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:18:59.026100 13017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:59.027365 13002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:18:59.027519 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "659dea626aad4321a96c89817ba2c751"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "e352b206d25245ccad056334d643b34c"
server_key_iv: "7563bb57fa94d45f1182470afc8ba3e0"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.027702 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:59.041075 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:59.041810 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:59.042095 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:59.058535 13002 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:44709
I20260929 02:18:59.058542 13068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:44709 every 8 connection(s)
I20260929 02:18:59.059799 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:59.061101 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13002
I20260929 02:18:59.061298 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:59.062038 7848 external_mini_cluster.cc:1428] Setting key c978982cf8786fe6872f491efc699966
I20260929 02:18:59.064301 13069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:59.075016 13072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:59.063836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34166 (local address 127.7.170.62:44709)
0929 02:18:59.064711 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:59.064728 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:59.064773 (+ 45us) server_negotiation.cc:407] Connection header received
0929 02:18:59.066374 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:59.066397 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:59.066838 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:59.067155 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:59.067752 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.069110 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.070927 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.071569 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.071697 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.072833 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:59.072840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:59.072853 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:59.073430 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:59.073506 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:59.073526 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:59.073611 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:59.073817 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:18:59.074114 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":280,"threads_started":1}
I20260929 02:18:59.075920 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751: Bootstrap starting.
I20260929 02:18:59.078481 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:18:59.079035 13069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:59.079918 13069 log.cc:824] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:59.082441 13069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751: No bootstrap required, opened a new log
I20260929 02:18:59.089318 13069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } }
I20260929 02:18:59.089639 13069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:59.089720 13069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 659dea626aad4321a96c89817ba2c751, State: Initialized, Role: FOLLOWER
I20260929 02:18:59.090215 13069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } }
I20260929 02:18:59.090427 13069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:59.090539 13069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:59.090683 13069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:59.092775 13069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } }
I20260929 02:18:59.093284 13069 leader_election.cc:302] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 659dea626aad4321a96c89817ba2c751; no voters:
I20260929 02:18:59.093798 13069 leader_election.cc:288] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:59.093942 13074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:59.094578 13074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [term 1 LEADER]: Becoming Leader. State: Replica: 659dea626aad4321a96c89817ba2c751, State: Running, Role: LEADER
I20260929 02:18:59.095059 13074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } }
I20260929 02:18:59.095611 13069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:59.098100 13075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "659dea626aad4321a96c89817ba2c751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } } }
I20260929 02:18:59.098518 13075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:59.098083 13076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 659dea626aad4321a96c89817ba2c751. Latest consensus state: current_term: 1 leader_uuid: "659dea626aad4321a96c89817ba2c751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "659dea626aad4321a96c89817ba2c751" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44709 } } }
I20260929 02:18:59.099366 13076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751 [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:59.106114 13081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:59.111064 13081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:59.120230 13081 catalog_manager.cc:1347] Generated new cluster ID: af35f2c637ce4abeaaff42146086e04b
I20260929 02:18:59.120394 13081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:59.142334 13081 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:59.144157 13081 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:59.160557 13081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 659dea626aad4321a96c89817ba2c751: Generated new TSK 0
I20260929 02:18:59.161403 13081 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:18:59.203375 13099 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260929 02:18:59.203549 13099 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:18:59.204403 13072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:59.183898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.7.170.62:44709)
0929 02:18:59.184092 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:59.184098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:59.184649 (+ 551us) server_negotiation.cc:407] Connection header received
0929 02:18:59.184976 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:59.184980 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:59.185046 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:59.185162 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:18:59.185739 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.186456 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.188501 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.188963 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.189061 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.189655 (+ 594us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:18:59.203896 (+ 14241us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:18:59.204167 (+ 271us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34176: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:18:59.204515 13072 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34176: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260929 02:18:59.206652 13098 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:18:59.183887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:44709 (local address 127.0.0.1:34176)
0929 02:18:59.184541 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:59.184564 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:59.184743 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:59.185221 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:59.185226 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:59.185239 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:18:59.185592 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:59.185604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.186576 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.186583 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:59.188345 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:59.188361 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.189110 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.189117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:59.189311 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.189402 (+ 91us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:18:59.204173 (+ 14771us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0929 02:18:59.204534 (+ 361us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.170.62:44709: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":310,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.170.62:44709: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44791/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260929 02:18:59.208521 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13002
I20260929 02:18:59.209633 13064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:59.302196 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13002
2026-09-29T02:18:59Z chronyd exiting
I20260929 02:18:59.349037 7848 test_util.cc:175] -----------------------------------------------
I20260929 02:18:59.349186 7848 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790648296482495-7848-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (715 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260929 02:18:59.352741 7848 mini_oidc.cc:152] Starting JWKS server
I20260929 02:18:59.354517 7848 webserver.cc:460] Webserver started at https://127.0.0.1:46793/ using document root <none> and password file <none>
I20260929 02:18:59.354720 7848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:18:59.355355 7848 webserver.cc:460] Webserver started at http://127.0.0.1:37869/ using document root <none> and password file <none>
2026-09-29T02:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:18:59Z Disabled control of system clock
I20260929 02:18:59.379137 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39105
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:40379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:39105
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46793/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/testchainca.cert with env {}
W20260929 02:18:59.680161 13106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:18:59.680471 13106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:18:59.680599 13106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:18:59.687860 13106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:18:59.687997 13106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:18:59.688058 13106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:18:59.688134 13106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:18:59.688189 13106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:18:59.688269 13106 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:18:59.688318 13106 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46793/jwks.json
I20260929 02:18:59.698325 13106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:39105
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46793/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:18:59.699642 13106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:18:59.700951 13106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:18:59.712016 13111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:59.711937 13114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:18:59.711973 13112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:18:59.713179 13106 server_base.cc:1034] running on GCE node
I20260929 02:18:59.713985 13106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:18:59.715337 13106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:18:59.716594 13106 hybrid_clock.cc:648] HybridClock initialized: now 1790648339716536 us; error 47 us; skew 500 ppm
I20260929 02:18:59.716988 13106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:18:59.719182 13106 webserver.cc:460] Webserver started at http://127.7.170.62:42437/ using document root <none> and password file <none>
I20260929 02:18:59.719866 13106 fs_manager.cc:359] Metadata directory not provided
I20260929 02:18:59.719975 13106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:18:59.720266 13106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:18:59.723112 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "b258ab57fd8f4008aa47483a0cfd63fa"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "ebc9eb9f97e455230ba44da2cdcc3af7"
server_key_iv: "fe0f548eefcd622ed8af2d1bb7ea3021"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.723862 13106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "b258ab57fd8f4008aa47483a0cfd63fa"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "ebc9eb9f97e455230ba44da2cdcc3af7"
server_key_iv: "fe0f548eefcd622ed8af2d1bb7ea3021"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.728820 13106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:18:59.732141 13120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:18:59.733600 13106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:18:59.733760 13106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "b258ab57fd8f4008aa47483a0cfd63fa"
format_stamp: "Formatted at 2026-09-29 02:18:59 on dist-test-slave-8cs2"
server_key: "ebc9eb9f97e455230ba44da2cdcc3af7"
server_key_iv: "fe0f548eefcd622ed8af2d1bb7ea3021"
server_key_version: "encryptionkey@0"
I20260929 02:18:59.733940 13106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:18:59.744151 13106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:18:59.744844 13106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:18:59.745146 13106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:18:59.761236 13106 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:39105
I20260929 02:18:59.761255 13172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:39105 every 8 connection(s)
I20260929 02:18:59.762503 13106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:18:59.763932 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13106
I20260929 02:18:59.764117 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:18:59.764691 7848 external_mini_cluster.cc:1428] Setting key c1e3c1b5bdce7f09218e6788e7e610dd
I20260929 02:18:59.767506 13173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:18:59.776901 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:59.766295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49132 (local address 127.7.170.62:39105)
0929 02:18:59.767132 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:59.767143 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:59.767183 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:18:59.768740 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:59.768761 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:59.769212 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:59.769547 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:18:59.770088 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.771368 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.772626 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.773295 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.773449 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.774650 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:18:59.774658 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:18:59.774671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:18:59.775209 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:18:59.775277 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:18:59.775299 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:18:59.775416 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:18:59.775637 (+ 221us) server_negotiation.cc:299] Negotiation successful
0929 02:18:59.775914 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":268,"threads_started":1}
I20260929 02:18:59.777721 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa: Bootstrap starting.
I20260929 02:18:59.780901 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:18:59.781328 13173 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:18:59.782274 13173 log.cc:824] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa: Log is configured to *not* fsync() on all Append() calls
I20260929 02:18:59.785099 13173 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa: No bootstrap required, opened a new log
I20260929 02:18:59.792672 13173 raft_consensus.cc:348] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } }
I20260929 02:18:59.793223 13173 raft_consensus.cc:374] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:18:59.793351 13173 raft_consensus.cc:729] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b258ab57fd8f4008aa47483a0cfd63fa, State: Initialized, Role: FOLLOWER
I20260929 02:18:59.793936 13173 consensus_queue.cc:260] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } }
I20260929 02:18:59.794138 13173 raft_consensus.cc:388] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:18:59.794257 13173 raft_consensus.cc:482] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:18:59.794423 13173 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:18:59.796463 13173 raft_consensus.cc:504] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } }
I20260929 02:18:59.796947 13173 leader_election.cc:302] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b258ab57fd8f4008aa47483a0cfd63fa; no voters:
I20260929 02:18:59.797580 13173 leader_election.cc:288] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:18:59.797716 13178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:18:59.798409 13178 raft_consensus.cc:686] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [term 1 LEADER]: Becoming Leader. State: Replica: b258ab57fd8f4008aa47483a0cfd63fa, State: Running, Role: LEADER
I20260929 02:18:59.798906 13178 consensus_queue.cc:237] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } }
I20260929 02:18:59.799515 13173 sys_catalog.cc:564] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:18:59.801517 13179 sys_catalog.cc:455] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } } }
I20260929 02:18:59.801496 13180 sys_catalog.cc:455] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader b258ab57fd8f4008aa47483a0cfd63fa. Latest consensus state: current_term: 1 leader_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b258ab57fd8f4008aa47483a0cfd63fa" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39105 } } }
I20260929 02:18:59.801973 13180 sys_catalog.cc:458] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:59.801973 13179 sys_catalog.cc:458] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa [sys.catalog]: This master's current role is: LEADER
I20260929 02:18:59.803820 13184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:18:59.808586 13184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:18:59.817313 13184 catalog_manager.cc:1347] Generated new cluster ID: 3078771045a341a9a930be407e55330d
I20260929 02:18:59.817408 13184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:18:59.829465 13184 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:18:59.831317 13184 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:18:59.845098 13184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b258ab57fd8f4008aa47483a0cfd63fa: Generated new TSK 0
I20260929 02:18:59.846052 13184 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260929 02:18:59.885450 13203 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260929 02:18:59.885553 13203 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260929 02:18:59.886814 13202 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:18:59.867391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39105 (local address 127.0.0.1:49140)
0929 02:18:59.867948 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0929 02:18:59.867963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:18:59.868073 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:18:59.868489 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:18:59.868492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:18:59.868501 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:18:59.868709 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:59.868717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.870122 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.870126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:59.871171 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:18:59.871181 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.872116 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.872121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:18:59.872232 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.872262 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:18:59.886362 (+ 14100us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:18:59.886550 (+ 188us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.170.62:39105: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":282,"threads_started":1}
I20260929 02:18:59.886811 13176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:18:59.867539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49140 (local address 127.7.170.62:39105)
0929 02:18:59.867716 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:18:59.867722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:18:59.868029 (+ 307us) server_negotiation.cc:407] Connection header received
0929 02:18:59.868224 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:18:59.868230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:18:59.868327 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:18:59.868486 (+ 159us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:18:59.868852 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.869991 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.871296 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:18:59.871970 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:18:59.872087 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:18:59.872497 (+ 410us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:18:59.886136 (+ 13639us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0929 02:18:59.886501 (+ 365us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49140: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 02:18:59.886916 13176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49140: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260929 02:18:59.887811 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13106
I20260929 02:18:59.888922 13167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:18:59.979069 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13106
2026-09-29T02:19:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (677 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260929 02:19:00.030546 7848 mini_oidc.cc:152] Starting JWKS server
I20260929 02:19:00.032297 7848 webserver.cc:460] Webserver started at https://127.0.0.1:35705/ using document root <none> and password file <none>
I20260929 02:19:00.032487 7848 mini_oidc.cc:189] Starting OIDC Discovery server
I20260929 02:19:00.033058 7848 webserver.cc:460] Webserver started at http://127.0.0.1:34447/ using document root <none> and password file <none>
2026-09-29T02:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:00Z Disabled control of system clock
I20260929 02:19:00.055496 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41447
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:39495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:41447
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:19:00.360246 13210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:00.360518 13210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:00.360622 13210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:00.367677 13210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:00.367776 13210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:00.367815 13210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:00.367895 13210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:00.367944 13210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:19:00.368026 13210 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:19:00.368072 13210 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35705/jwks.json
I20260929 02:19:00.377084 13210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:41447
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:00.378336 13210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:00.380057 13210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:00.390847 13215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:00.390893 13218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:00.390797 13216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:00.391673 13210 server_base.cc:1034] running on GCE node
I20260929 02:19:00.392344 13210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:00.393651 13210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:00.394838 13210 hybrid_clock.cc:648] HybridClock initialized: now 1790648340394801 us; error 35 us; skew 500 ppm
I20260929 02:19:00.395174 13210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:00.397238 13210 webserver.cc:460] Webserver started at http://127.7.170.62:42261/ using document root <none> and password file <none>
I20260929 02:19:00.397874 13210 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:00.397974 13210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:00.398295 13210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:00.401264 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "cd2050712c314948acb9b9d650a1a3a4"
format_stamp: "Formatted at 2026-09-29 02:19:00 on dist-test-slave-8cs2"
server_key: "9f1964bfc72571411c82ecf7ab4035e8"
server_key_iv: "eda123c1dc2885ebbc689399594f61a4"
server_key_version: "encryptionkey@0"
I20260929 02:19:00.401996 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "cd2050712c314948acb9b9d650a1a3a4"
format_stamp: "Formatted at 2026-09-29 02:19:00 on dist-test-slave-8cs2"
server_key: "9f1964bfc72571411c82ecf7ab4035e8"
server_key_iv: "eda123c1dc2885ebbc689399594f61a4"
server_key_version: "encryptionkey@0"
I20260929 02:19:00.406677 13210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:19:00.409844 13224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:00.411154 13210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:19:00.411298 13210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "cd2050712c314948acb9b9d650a1a3a4"
format_stamp: "Formatted at 2026-09-29 02:19:00 on dist-test-slave-8cs2"
server_key: "9f1964bfc72571411c82ecf7ab4035e8"
server_key_iv: "eda123c1dc2885ebbc689399594f61a4"
server_key_version: "encryptionkey@0"
I20260929 02:19:00.411470 13210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:00.426637 13210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:00.427408 13210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:00.427670 13210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:00.444137 13210 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:41447
I20260929 02:19:00.444160 13276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:41447 every 8 connection(s)
I20260929 02:19:00.445402 13210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:00.448899 13277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:00.450292 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13210
I20260929 02:19:00.450462 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:00.451072 7848 external_mini_cluster.cc:1428] Setting key b5334e95ed0f5b6b36a8c6dd816a1fc2
I20260929 02:19:00.461611 13277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4: Bootstrap starting.
I20260929 02:19:00.465204 13277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:00.465915 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:00.452812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.7.170.62:41447)
0929 02:19:00.453914 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:00.453929 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:00.453993 (+ 64us) server_negotiation.cc:407] Connection header received
0929 02:19:00.456211 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:00.456237 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:00.456815 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:00.457257 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:00.457787 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.459779 (+ 1992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.461050 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.462049 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.462228 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:00.463244 (+ 1016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:00.463253 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:00.463276 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:00.463946 (+ 670us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:00.464069 (+ 123us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:00.464095 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:00.464212 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:00.464343 (+ 131us) server_negotiation.cc:299] Negotiation successful
0929 02:19:00.464642 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":348,"threads_started":1}
I20260929 02:19:00.466463 13277 log.cc:824] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:00.469446 13277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4: No bootstrap required, opened a new log
I20260929 02:19:00.469616 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:19:00.477064 13277 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } }
I20260929 02:19:00.477418 13277 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:00.477537 13277 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd2050712c314948acb9b9d650a1a3a4, State: Initialized, Role: FOLLOWER
I20260929 02:19:00.478098 13277 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } }
I20260929 02:19:00.478304 13277 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:00.478422 13277 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:00.478572 13277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:00.480661 13277 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } }
I20260929 02:19:00.481192 13277 leader_election.cc:302] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd2050712c314948acb9b9d650a1a3a4; no voters:
I20260929 02:19:00.481829 13277 leader_election.cc:288] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:00.481930 13282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:00.482584 13282 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [term 1 LEADER]: Becoming Leader. State: Replica: cd2050712c314948acb9b9d650a1a3a4, State: Running, Role: LEADER
I20260929 02:19:00.483122 13282 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } }
I20260929 02:19:00.483777 13277 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:00.485695 13283 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd2050712c314948acb9b9d650a1a3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } } }
I20260929 02:19:00.486115 13283 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:00.486418 13284 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd2050712c314948acb9b9d650a1a3a4. Latest consensus state: current_term: 1 leader_uuid: "cd2050712c314948acb9b9d650a1a3a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd2050712c314948acb9b9d650a1a3a4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41447 } } }
I20260929 02:19:00.486790 13284 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:00.488209 13291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:00.492952 13291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:00.500502 13291 catalog_manager.cc:1347] Generated new cluster ID: 6b521bb7b8eb4e52b58bf8f17c52448e
I20260929 02:19:00.500607 13291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:00.510959 13291 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:00.512720 13291 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:00.521353 13291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd2050712c314948acb9b9d650a1a3a4: Generated new TSK 0
I20260929 02:19:00.522413 13291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:19:00.546546 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:00.538470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41664 (local address 127.7.170.62:41447)
0929 02:19:00.538661 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:00.538668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:00.538964 (+ 296us) server_negotiation.cc:407] Connection header received
0929 02:19:00.539169 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:00.539175 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:00.539270 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:00.539457 (+ 187us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:19:00.539889 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.541017 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.542307 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.542978 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.543100 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:00.543553 (+ 453us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:19:00.545912 (+ 2359us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:19:00.546239 (+ 327us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 02:19:00.546679 13280 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41664: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260929 02:19:00.548168 13306 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:00.538316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:41447 (local address 127.0.0.1:41664)
0929 02:19:00.538867 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:00.538883 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:00.538999 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:00.539493 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:00.539497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:00.539508 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:19:00.539742 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:00.539750 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.541143 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.541147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:00.542167 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:00.542177 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:00.543132 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:00.543137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:00.543272 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:00.543302 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:19:00.546118 (+ 2816us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0929 02:19:00.547766 (+ 1648us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.170.62:41447: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":1354,"thread_start_us":284,"threads_started":1}
I20260929 02:19:00.549240 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13210
I20260929 02:19:00.550225 13272 generic_service.cc:196] Checking for leaks (request via RPC)
E20260929 02:19:00.562103 13307 webserver.cc:533] Webserver: sslize failed:
E20260929 02:19:00.562234 13307 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260929 02:19:00.641080 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13210
2026-09-29T02:19:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (661 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-29T02:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:00Z Disabled control of system clock
I20260929 02:19:00.717156 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:46381
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:36701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:46381
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/testchainca.cert with env {}
W20260929 02:19:01.026839 13312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:01.027117 13312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:01.027225 13312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:01.034358 13312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:01.034456 13312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:01.034500 13312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:01.034572 13312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:01.034626 13312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260929 02:19:01.034684 13312 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260929 02:19:01.034734 13312 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260929 02:19:01.044490 13312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:46381
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:46381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:01.045769 13312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:01.047062 13312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:01.058238 13317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:01.058209 13320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:01.058266 13318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:01.059019 13312 server_base.cc:1034] running on GCE node
I20260929 02:19:01.059753 13312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:01.061113 13312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:01.062328 13312 hybrid_clock.cc:648] HybridClock initialized: now 1790648341062282 us; error 34 us; skew 500 ppm
I20260929 02:19:01.062666 13312 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:01.064765 13312 webserver.cc:460] Webserver started at http://127.7.170.62:35817/ using document root <none> and password file <none>
I20260929 02:19:01.065457 13312 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:01.065557 13312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:01.065853 13312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:01.068657 13312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e6fca20e3b344ca6b44a4fd3d909f500"
format_stamp: "Formatted at 2026-09-29 02:19:01 on dist-test-slave-8cs2"
server_key: "6c5d276d3fa7b0153005653f3a44fa48"
server_key_iv: "ff49c7301bbffb0f0c9f2221607241de"
server_key_version: "encryptionkey@0"
I20260929 02:19:01.069427 13312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e6fca20e3b344ca6b44a4fd3d909f500"
format_stamp: "Formatted at 2026-09-29 02:19:01 on dist-test-slave-8cs2"
server_key: "6c5d276d3fa7b0153005653f3a44fa48"
server_key_iv: "ff49c7301bbffb0f0c9f2221607241de"
server_key_version: "encryptionkey@0"
I20260929 02:19:01.074360 13312 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260929 02:19:01.077775 13326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:01.079171 13312 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 02:19:01.079330 13312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e6fca20e3b344ca6b44a4fd3d909f500"
format_stamp: "Formatted at 2026-09-29 02:19:01 on dist-test-slave-8cs2"
server_key: "6c5d276d3fa7b0153005653f3a44fa48"
server_key_iv: "ff49c7301bbffb0f0c9f2221607241de"
server_key_version: "encryptionkey@0"
I20260929 02:19:01.079528 13312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:01.092813 13312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:01.093706 13312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:01.093961 13312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:01.110601 13312 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:46381
I20260929 02:19:01.110621 13377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:46381 every 8 connection(s)
I20260929 02:19:01.111930 13312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:01.113430 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13312
I20260929 02:19:01.113605 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:01.114429 7848 external_mini_cluster.cc:1428] Setting key 46770d47158d9a3f1a2f4f15106ed062
I20260929 02:19:01.116577 13378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:01.127322 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:01.116279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46512 (local address 127.7.170.62:46381)
0929 02:19:01.117236 (+ 957us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:01.117247 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:01.117288 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:19:01.118844 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:01.118868 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:01.119355 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:01.119717 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:01.120261 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.121660 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.122906 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.123602 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.123770 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:01.124865 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:01.124880 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:01.124896 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:01.125490 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:01.125572 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:01.125592 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:01.125690 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:01.125906 (+ 216us) server_negotiation.cc:299] Negotiation successful
0929 02:19:01.126174 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":493,"thread_start_us":289,"threads_started":1}
I20260929 02:19:01.127677 13378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500: Bootstrap starting.
I20260929 02:19:01.131517 13378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:01.131619 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:19:01.132479 13378 log.cc:824] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:01.135521 13378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500: No bootstrap required, opened a new log
I20260929 02:19:01.142536 13378 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } }
I20260929 02:19:01.142874 13378 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:01.142973 13378 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6fca20e3b344ca6b44a4fd3d909f500, State: Initialized, Role: FOLLOWER
I20260929 02:19:01.143608 13378 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } }
I20260929 02:19:01.143883 13378 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:01.144058 13378 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:01.144197 13378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:01.146286 13378 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } }
I20260929 02:19:01.146739 13378 leader_election.cc:302] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6fca20e3b344ca6b44a4fd3d909f500; no voters:
I20260929 02:19:01.147256 13378 leader_election.cc:288] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:01.147397 13383 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:01.148052 13383 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [term 1 LEADER]: Becoming Leader. State: Replica: e6fca20e3b344ca6b44a4fd3d909f500, State: Running, Role: LEADER
I20260929 02:19:01.148525 13383 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } }
I20260929 02:19:01.149163 13378 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:01.151151 13384 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } } }
I20260929 02:19:01.151561 13384 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:01.151145 13385 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6fca20e3b344ca6b44a4fd3d909f500. Latest consensus state: current_term: 1 leader_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6fca20e3b344ca6b44a4fd3d909f500" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 46381 } } }
I20260929 02:19:01.152611 13385 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:01.154039 13389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:01.158927 13389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:01.167553 13389 catalog_manager.cc:1347] Generated new cluster ID: 63faa96d0056429aa36cf0a0c8675a1e
I20260929 02:19:01.167650 13389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:01.222601 13389 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:01.223868 13389 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:01.231916 13389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6fca20e3b344ca6b44a4fd3d909f500: Generated new TSK 0
I20260929 02:19:01.232733 13389 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:19:01.280591 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:01.246929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.7.170.62:46381)
0929 02:19:01.247104 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:01.247111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:01.247471 (+ 360us) server_negotiation.cc:407] Connection header received
0929 02:19:01.247702 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:01.247706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:01.247778 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:01.247905 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0929 02:19:01.248356 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.249165 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.250583 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.251066 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.251170 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:01.251641 (+ 471us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0929 02:19:01.279874 (+ 28233us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:19:01.280290 (+ 416us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46514: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:19:01.280726 13381 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46514: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260929 02:19:01.280845 13408 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:01.246827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:46381 (local address 127.0.0.1:46514)
0929 02:19:01.247371 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:01.247386 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:01.247527 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:01.247933 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:01.247938 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:01.247950 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0929 02:19:01.248216 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:01.248224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.249301 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.249305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:01.250445 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:01.250457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:01.251219 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:01.251225 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:01.251381 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:01.251413 (+ 32us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0929 02:19:01.280184 (+ 28771us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0929 02:19:01.280439 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.170.62:46381: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":292,"threads_started":1}
I20260929 02:19:01.282550 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13312
I20260929 02:19:01.283691 13373 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:01.380611 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13312
2026-09-29T02:19:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (718 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (387 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (393 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:02 dist-test-slave-8cs2 krb5kdc[13423](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:02 dist-test-slave-8cs2 krb5kdc[13423](info): set up 2 sockets
Sep 29 02:19:02 dist-test-slave-8cs2 krb5kdc[13423](info): commencing operation
krb5kdc: starting...
W20260929 02:19:03.165663 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.971s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:03 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:19:03.352438 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34417
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:34417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:03.656812 13436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:03.657107 13436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:03.657229 13436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:03.664287 13436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:03.664387 13436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:03.664458 13436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:03.664525 13436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:03.664582 13436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:03.674098 13436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:34417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:03.675369 13436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:03.676615 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:03.687834 13442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:03.687799 13444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:03.687866 13441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:03.688467 13436 server_base.cc:1034] running on GCE node
I20260929 02:19:03.689296 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:03.690661 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:03.691870 13436 hybrid_clock.cc:648] HybridClock initialized: now 1790648343691819 us; error 42 us; skew 500 ppm
I20260929 02:19:03.692184 13436 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:03 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648343, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:03.695106 13436 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:19:03.696491 13436 webserver.cc:460] Webserver started at http://127.7.170.62:41827/ using document root <none> and password file <none>
I20260929 02:19:03.697160 13436 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:03.697259 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:03.697525 13436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:03.700435 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "4112c57d70d344b7b76e36cf4342b111"
format_stamp: "Formatted at 2026-09-29 02:19:03 on dist-test-slave-8cs2"
server_key: "657434235c8fb0397ef9cf58311a9704"
server_key_iv: "9623cf59acf1be1f5e3c54e62d186e81"
server_key_version: "encryptionkey@0"
I20260929 02:19:03.701202 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "4112c57d70d344b7b76e36cf4342b111"
format_stamp: "Formatted at 2026-09-29 02:19:03 on dist-test-slave-8cs2"
server_key: "657434235c8fb0397ef9cf58311a9704"
server_key_iv: "9623cf59acf1be1f5e3c54e62d186e81"
server_key_version: "encryptionkey@0"
I20260929 02:19:03.706306 13436 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:19:03.709544 13451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:03.710799 13436 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:19:03.710951 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "4112c57d70d344b7b76e36cf4342b111"
format_stamp: "Formatted at 2026-09-29 02:19:03 on dist-test-slave-8cs2"
server_key: "657434235c8fb0397ef9cf58311a9704"
server_key_iv: "9623cf59acf1be1f5e3c54e62d186e81"
server_key_version: "encryptionkey@0"
I20260929 02:19:03.711138 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:03.732543 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:03.736224 13436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:03.736465 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:03.752182 13436 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:34417
I20260929 02:19:03.752205 13502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:34417 every 8 connection(s)
I20260929 02:19:03.753423 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:03.756839 13503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:03.760028 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13436
I20260929 02:19:03.760210 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:03.760784 7848 external_mini_cluster.cc:1428] Setting key 4f5e1e0976a59a1354d3e5721b30bd2e
I20260929 02:19:03.768741 13503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111: Bootstrap starting.
I20260929 02:19:03.772377 13503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:03.773655 13503 log.cc:824] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:19:03 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:03.776460 13503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111: No bootstrap required, opened a new log
I20260929 02:19:03.782056 13506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:03.762604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38036 (local address 127.7.170.62:34417)
0929 02:19:03.763319 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:03.763331 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:03.763370 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:19:03.764888 (+ 1518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:03.764909 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:03.765361 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:03.765686 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:03.767273 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:03.768519 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:03.770154 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:03.770831 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:03.770954 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:03.775455 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:03.775462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:03.775478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:03.775514 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:03.778325 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:03.778944 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:03.778950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:03.778956 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:03.779119 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:03.779450 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:03.779453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:03.779455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:03.779942 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:03.780218 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:03.780642 (+ 424us) server_negotiation.cc:299] Negotiation successful
0929 02:19:03.781062 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":336,"threads_started":1}
I20260929 02:19:03.784219 13503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } }
I20260929 02:19:03.784615 13503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:03.784721 13503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4112c57d70d344b7b76e36cf4342b111, State: Initialized, Role: FOLLOWER
I20260929 02:19:03.785303 13503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } }
I20260929 02:19:03.785508 13503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:03.785626 13503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:03.785768 13503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:03.788116 13503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } }
I20260929 02:19:03.788550 13503 leader_election.cc:302] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4112c57d70d344b7b76e36cf4342b111; no voters:
I20260929 02:19:03.789084 13503 leader_election.cc:288] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:03.789186 13508 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:03.789747 13508 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [term 1 LEADER]: Becoming Leader. State: Replica: 4112c57d70d344b7b76e36cf4342b111, State: Running, Role: LEADER
I20260929 02:19:03.790230 13508 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } }
I20260929 02:19:03.790926 13503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:03.792768 13509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4112c57d70d344b7b76e36cf4342b111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } } }
I20260929 02:19:03.793154 13509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:03.793124 13510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4112c57d70d344b7b76e36cf4342b111. Latest consensus state: current_term: 1 leader_uuid: "4112c57d70d344b7b76e36cf4342b111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4112c57d70d344b7b76e36cf4342b111" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34417 } } }
I20260929 02:19:03.794596 13510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:03.796072 13515 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:03.800384 13515 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:03.807849 13515 catalog_manager.cc:1347] Generated new cluster ID: 09301548569b4fec8c8a45e48515cfd0
I20260929 02:19:03.807951 13515 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:03.825553 13515 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:03.826761 13515 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:03.834205 13515 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4112c57d70d344b7b76e36cf4342b111: Generated new TSK 0
I20260929 02:19:03.835201 13515 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:19:03.889705 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:04.196522 13530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:04.196767 13530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:04.196877 13530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:04.203840 13530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:04.203933 13530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:04.204047 13530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:04.213411 13530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:04.214596 13530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:04.215837 13530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:04.227676 13538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:04.227720 13535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:04.227746 13536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:04.228561 13530 server_base.cc:1034] running on GCE node
I20260929 02:19:04.229152 13530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:04.229975 13530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:04.231169 13530 hybrid_clock.cc:648] HybridClock initialized: now 1790648344231126 us; error 36 us; skew 500 ppm
I20260929 02:19:04.231451 13530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:04 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648344, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:04.240816 13530 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:19:04.242198 13530 webserver.cc:460] Webserver started at http://127.7.170.1:36261/ using document root <none> and password file <none>
I20260929 02:19:04.242803 13530 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:04.242900 13530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:04.243144 13530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:04.245891 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "a964b4d13a6649d9af073577ac56e05f"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "9ebe9348f8bad6d3566bb2688172488e"
server_key_iv: "e752f4b38005735e0b5e71072c9b32eb"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.246555 13530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "a964b4d13a6649d9af073577ac56e05f"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "9ebe9348f8bad6d3566bb2688172488e"
server_key_iv: "e752f4b38005735e0b5e71072c9b32eb"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.251281 13530 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:04.254490 13545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.255712 13530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:19:04.255842 13530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "a964b4d13a6649d9af073577ac56e05f"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "9ebe9348f8bad6d3566bb2688172488e"
server_key_iv: "e752f4b38005735e0b5e71072c9b32eb"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.256022 13530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:04.279943 13530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:04.283576 13530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:04.283807 13530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:04.284618 13530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:04.285832 13530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:04.285900 13530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.286016 13530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:04.286068 13530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.314399 13530 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:34521
I20260929 02:19:04.314432 13657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:34521 every 8 connection(s)
I20260929 02:19:04.315691 13530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:04.318174 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13530
I20260929 02:19:04.318346 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:04.319043 7848 external_mini_cluster.cc:1428] Setting key b494b962d290fcf97c419842ab5862a4
Sep 29 02:19:04 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648344, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:04.335793 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:04.318204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:58509 (local address 127.7.170.62:34417)
0929 02:19:04.318663 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:04.318670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:04.319923 (+ 1253us) server_negotiation.cc:407] Connection header received
0929 02:19:04.321784 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:04.321790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:04.321867 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:04.321970 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:04.323367 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.324366 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.325940 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.326459 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.326583 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:04.330270 (+ 3687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:04.330275 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:04.330278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:04.330311 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:04.332710 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:04.333552 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:04.333556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:04.333558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:04.333671 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:04.334147 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:04.334151 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:04.334153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:04.334447 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:04.334563 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:04.335371 (+ 808us) server_negotiation.cc:299] Negotiation successful
0929 02:19:04.335543 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":215,"threads_started":1}
I20260929 02:19:04.337303 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:04.318766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34417 (local address 127.7.170.1:58509)
0929 02:19:04.319559 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:04.319605 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:04.321513 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:04.322178 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:04.322185 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:04.322651 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:04.323207 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:04.323225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.324506 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.324511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:04.325807 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:04.325818 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.326640 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.326645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:04.326841 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:04.327481 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:04.327508 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:04.330051 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:04.332847 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:04.332856 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:04.332875 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:04.333450 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:04.333762 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:04.333766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:04.333770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:04.334030 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:04.334561 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:04.334568 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:04.334972 (+ 404us) client_negotiation.cc:769] Sending connection context
0929 02:19:04.335294 (+ 322us) client_negotiation.cc:240] Negotiation successful
0929 02:19:04.335829 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":323,"threads_started":1}
I20260929 02:19:04.338724 13658 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34417
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:19:04.339012 13658 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:04.339695 13658 heartbeater.cc:511] Master 127.7.170.62:34417 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:19:04.341656 13468 ts_manager.cc:194] Registered new tserver with Master: a964b4d13a6649d9af073577ac56e05f (127.7.170.1:34521)
I20260929 02:19:04.343554 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:58509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:19:04.363893 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:04.664378 13664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:04.664659 13664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:04.664757 13664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:04.671838 13664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:04.671947 13664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:04.672060 13664 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:04.681840 13664 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:04.683059 13664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:04.684305 13664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:04.696980 13669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:04.697026 13672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:04.697045 13670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:04.697872 13664 server_base.cc:1034] running on GCE node
I20260929 02:19:04.698515 13664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:04.699402 13664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:04.700585 13664 hybrid_clock.cc:648] HybridClock initialized: now 1790648344700531 us; error 48 us; skew 500 ppm
I20260929 02:19:04.700867 13664 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:04 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648344, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:04.703770 13664 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:19:04.705090 13664 webserver.cc:460] Webserver started at http://127.7.170.2:40465/ using document root <none> and password file <none>
I20260929 02:19:04.705705 13664 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:04.705813 13664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:04.706067 13664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:04.708819 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "90cbb0c0022f44b6851c1eb818a6153e"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "199109b57e01a21a86bbbba926d40946"
server_key_iv: "fbce1c14fd7d50f397cb2d6106e1f283"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.709758 13664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "90cbb0c0022f44b6851c1eb818a6153e"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "199109b57e01a21a86bbbba926d40946"
server_key_iv: "fbce1c14fd7d50f397cb2d6106e1f283"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.714682 13664 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:19:04.718209 13679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.719556 13664 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:19:04.719703 13664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "90cbb0c0022f44b6851c1eb818a6153e"
format_stamp: "Formatted at 2026-09-29 02:19:04 on dist-test-slave-8cs2"
server_key: "199109b57e01a21a86bbbba926d40946"
server_key_iv: "fbce1c14fd7d50f397cb2d6106e1f283"
server_key_version: "encryptionkey@0"
I20260929 02:19:04.719885 13664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:04.734431 13664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:04.739099 13664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:04.739359 13664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:04.740305 13664 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:04.741714 13664 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:04.741782 13664 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.741868 13664 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:04.741916 13664 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:04.771884 13664 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44279
I20260929 02:19:04.771924 13791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44279 every 8 connection(s)
I20260929 02:19:04.773145 13664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:04.779867 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13664
I20260929 02:19:04.780017 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:04.780573 7848 external_mini_cluster.cc:1428] Setting key 33bb239f542b8830ac9191830cfe236c
Sep 29 02:19:04 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648344, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:04.792208 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:04.775578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:55655 (local address 127.7.170.62:34417)
0929 02:19:04.775767 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:04.775773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:04.776950 (+ 1177us) server_negotiation.cc:407] Connection header received
0929 02:19:04.778645 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:04.778649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:04.778713 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:04.778797 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:04.780224 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.781275 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.783017 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.783401 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.783515 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:04.786677 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:04.786682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:04.786685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:04.786716 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:04.789303 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:04.790060 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:04.790064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:04.790066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:04.790203 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:04.790659 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:04.790662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:04.790664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:04.790925 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:04.791037 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:04.791799 (+ 762us) server_negotiation.cc:299] Negotiation successful
0929 02:19:04.791978 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:19:04.793592 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:04.776090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34417 (local address 127.7.170.2:55655)
0929 02:19:04.776784 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:04.776820 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:04.778410 (+ 1590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:04.779001 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:04.779008 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:04.779471 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:04.780053 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:04.780077 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.781421 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.781425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:04.782885 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:04.782897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:04.783603 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:04.783609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:04.783824 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:04.784408 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:04.784431 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:04.786476 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:04.789425 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:04.789434 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:04.789450 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:04.789939 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:04.790302 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:04.790307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:04.790311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:04.790567 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:04.791041 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:04.791049 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:04.791457 (+ 408us) client_negotiation.cc:769] Sending connection context
0929 02:19:04.791741 (+ 284us) client_negotiation.cc:240] Negotiation successful
0929 02:19:04.792250 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":290,"threads_started":1}
I20260929 02:19:04.794917 13792 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34417
I20260929 02:19:04.795186 13792 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:04.795912 13792 heartbeater.cc:511] Master 127.7.170.62:34417 requested a full tablet report, sending...
I20260929 02:19:04.797349 13468 ts_manager.cc:194] Registered new tserver with Master: 90cbb0c0022f44b6851c1eb818a6153e (127.7.170.2:44279)
I20260929 02:19:04.798187 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:55655
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:19:04.824424 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:05.133754 13797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:05.134021 13797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:05.134116 13797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:05.141557 13797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:05.141682 13797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:05.141865 13797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:05.151556 13797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34417
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:05.152698 13797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:05.153939 13797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:05.165732 13805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:05.165865 13802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:05.165964 13803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:05.166965 13797 server_base.cc:1034] running on GCE node
I20260929 02:19:05.167484 13797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:05.168291 13797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:05.169481 13797 hybrid_clock.cc:648] HybridClock initialized: now 1790648345169425 us; error 41 us; skew 500 ppm
I20260929 02:19:05.169747 13797 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.172430 13797 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:19:05.174021 13797 webserver.cc:460] Webserver started at http://127.7.170.3:43749/ using document root <none> and password file <none>
I20260929 02:19:05.174633 13797 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:05.174743 13797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:05.175016 13797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:05.177734 13797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "cd6eaf8f86734d5d80431be1ed99087f"
format_stamp: "Formatted at 2026-09-29 02:19:05 on dist-test-slave-8cs2"
server_key: "018e633c8cb0f77631d1514e78c1a9b0"
server_key_iv: "8baa9639f5fb7bab5467a2742810a0af"
server_key_version: "encryptionkey@0"
I20260929 02:19:05.178467 13797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "cd6eaf8f86734d5d80431be1ed99087f"
format_stamp: "Formatted at 2026-09-29 02:19:05 on dist-test-slave-8cs2"
server_key: "018e633c8cb0f77631d1514e78c1a9b0"
server_key_iv: "8baa9639f5fb7bab5467a2742810a0af"
server_key_version: "encryptionkey@0"
I20260929 02:19:05.183076 13797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:05.186168 13812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:05.187364 13797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:19:05.187502 13797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "cd6eaf8f86734d5d80431be1ed99087f"
format_stamp: "Formatted at 2026-09-29 02:19:05 on dist-test-slave-8cs2"
server_key: "018e633c8cb0f77631d1514e78c1a9b0"
server_key_iv: "8baa9639f5fb7bab5467a2742810a0af"
server_key_version: "encryptionkey@0"
I20260929 02:19:05.187666 13797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:05.208989 13797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:05.212779 13797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:05.213076 13797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:05.213994 13797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:05.215376 13797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:05.215466 13797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:05.215564 13797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:05.215621 13797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:05.244963 13797 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36959
I20260929 02:19:05.245035 13924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36959 every 8 connection(s)
I20260929 02:19:05.246194 13797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:05.250890 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13797
I20260929 02:19:05.251035 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:05.251430 7848 external_mini_cluster.cc:1428] Setting key 2ba44916a69add5c1bfb7b6452eb839a
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:05.264166 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.248509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:48193 (local address 127.7.170.62:34417)
0929 02:19:05.248673 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.248678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.249843 (+ 1165us) server_negotiation.cc:407] Connection header received
0929 02:19:05.251661 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.251667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.251741 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.251822 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.253394 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.254196 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.255648 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.256051 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.256142 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.259239 (+ 3097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.259244 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.259248 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.259279 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.261624 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.262278 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.262281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.262284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.262403 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.262759 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.262762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.262764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.263034 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.263144 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.263742 (+ 598us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.263910 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:19:05.265326 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.249017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34417 (local address 127.7.170.3:48193)
0929 02:19:05.249694 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:05.249737 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:05.251393 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:05.252064 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:05.252073 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:05.252547 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:05.253222 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:05.253250 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.254335 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.254339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:05.255523 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:05.255534 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.256210 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.256214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:05.256400 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.257024 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:05.257052 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:05.259063 (+ 2011us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:05.261748 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:05.261753 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:05.261766 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:05.262170 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:05.262477 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:05.262481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:05.262485 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:05.262671 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:05.263130 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:05.263135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:05.263440 (+ 305us) client_negotiation.cc:769] Sending connection context
0929 02:19:05.263671 (+ 231us) client_negotiation.cc:240] Negotiation successful
0929 02:19:05.264072 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":271,"threads_started":1}
I20260929 02:19:05.266654 13925 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34417
I20260929 02:19:05.266912 13925 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:05.267560 13925 heartbeater.cc:511] Master 127.7.170.62:34417 requested a full tablet report, sending...
I20260929 02:19:05.268812 13468 ts_manager.cc:194] Registered new tserver with Master: cd6eaf8f86734d5d80431be1ed99087f (127.7.170.3:36959)
I20260929 02:19:05.269594 13468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:48193
I20260929 02:19:05.278640 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/test-admin.keytab.
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.304225 7848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260929 02:19:05.308568 13932 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:05.306687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39306 (local address 127.7.170.1:34521)
0929 02:19:05.307208 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.307216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.307241 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:19:05.307303 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.307309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.307519 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.307643 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.308316 (+ 673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39306: BlockingRecv error: recv got EOF from 127.0.0.1:39306 (error 108)
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":297,"threads_started":1}
W20260929 02:19:05.308806 7848 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.310676 7848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.311928 7848 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:19:05.327315 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.313539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39314 (local address 127.7.170.1:34521)
0929 02:19:05.313769 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.313773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.313861 (+ 88us) server_negotiation.cc:407] Connection header received
0929 02:19:05.314020 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.314024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.314078 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.314156 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.315363 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.316512 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.318323 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.318851 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.318971 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.322416 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.322426 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.322439 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.322471 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.325078 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.325616 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.325621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.325624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.325732 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.326036 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.326039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.326041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.326352 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.326557 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.326825 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.327050 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260929 02:19:05.332116 13932 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:05.330836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.7.170.1:34521)
0929 02:19:05.331021 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.331026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.331189 (+ 163us) server_negotiation.cc:407] Connection header received
0929 02:19:05.331293 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.331297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.331347 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.331417 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.331932 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39316: BlockingRecv error: recv got EOF from 127.0.0.1:39316 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 02:19:05.332554 7848 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc)
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.333985 7848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.335134 7848 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:19:05.346493 13658 heartbeater.cc:503] Master 127.7.170.62:34417 was elected leader, sending a full tablet report...
I20260929 02:19:05.347820 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.336245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39332 (local address 127.7.170.1:34521)
0929 02:19:05.336400 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.336404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.336771 (+ 367us) server_negotiation.cc:407] Connection header received
0929 02:19:05.336885 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.336888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.336971 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.337044 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.337884 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.338563 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.339644 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.340068 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.340154 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.343340 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.343345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.343348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.343384 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.345590 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.346131 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.346135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.346137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.346235 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.346580 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.346584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.346587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.346876 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.347063 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.347351 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.347549 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260929 02:19:05.351426 13932 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:05.349968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.7.170.1:34521)
0929 02:19:05.350125 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.350130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.350318 (+ 188us) server_negotiation.cc:407] Connection header received
0929 02:19:05.350446 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.350451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.350531 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.350602 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.351226 (+ 624us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39344: BlockingRecv error: recv got EOF from 127.0.0.1:39344 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260929 02:19:05.351583 7848 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790648296482495-7848-0/krb5kdc/krb5cc' of type FILE
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.353117 7848 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:05.354266 7848 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:19:05.366421 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.355425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39354 (local address 127.7.170.1:34521)
0929 02:19:05.355582 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.355587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.355746 (+ 159us) server_negotiation.cc:407] Connection header received
0929 02:19:05.355906 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.355910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.355961 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.356031 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.356857 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.357581 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.358762 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.359129 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.359226 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.362284 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.362289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.362292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.362325 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.364369 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.364867 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.364870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.364872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.365035 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.365300 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.365303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.365305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.365555 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.365770 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.365977 (+ 207us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.366126 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:19:05.367718 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13530
I20260929 02:19:05.377708 13932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.368308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.7.170.1:34521)
0929 02:19:05.368428 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.368432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.368740 (+ 308us) server_negotiation.cc:407] Connection header received
0929 02:19:05.368861 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.368864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.368911 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.369016 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.369757 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.370381 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.371436 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.371825 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.371907 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.373571 (+ 1664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.373575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.373578 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.373602 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.375736 (+ 2134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.376260 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.376264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.376266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.376379 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.376655 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.376658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.376659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.376875 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.377109 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.377324 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.377468 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:19:05.378175 13653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:05.458387 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13530
I20260929 02:19:05.471205 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13664
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:19:05.484508 13945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.472027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.7.170.2:44279)
0929 02:19:05.472512 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.472519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.472544 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:19:05.472606 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.472613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.472812 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.472962 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.473777 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.474744 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.475869 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.476407 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.476503 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.479844 (+ 3341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.479854 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.479868 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.479898 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.482379 (+ 2481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.482913 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.482919 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.482922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.483039 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.483323 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.483325 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.483327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.483626 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.483837 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.484056 (+ 219us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.484233 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":231,"threads_started":1}
I20260929 02:19:05.486099 13787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:05.564733 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13664
I20260929 02:19:05.576884 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13797
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13423](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:19:05.592047 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:05.577883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.7.170.3:36959)
0929 02:19:05.578467 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:05.578473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:05.578512 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:19:05.578571 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:05.578578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:05.578783 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:05.578901 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:05.579786 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.580794 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.582407 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:05.582972 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:05.583070 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:05.586609 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:05.586621 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:05.586636 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:05.586673 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:05.589461 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.590074 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.590080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.590083 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.590200 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:05.590534 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:05.590538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:05.590540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:05.590875 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:05.591190 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:05.591501 (+ 311us) server_negotiation.cc:299] Negotiation successful
0929 02:19:05.591771 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":298,"threads_started":1}
I20260929 02:19:05.593248 13920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:05.673568 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13797
I20260929 02:19:05.686019 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13436
I20260929 02:19:05.687098 13498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:05.785434 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13436
2026-09-29T02:19:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3627 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13952](info): set up 2 sockets
Sep 29 02:19:05 dist-test-slave-8cs2 krb5kdc[13952](info): commencing operation
krb5kdc: starting...
W20260929 02:19:06.785676 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:06 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab.
Entry for principal oryx/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab.
I20260929 02:19:06.978467 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33267
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab
--principal=oryx/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:33267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:07.286319 13965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:07.286597 13965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:07.286723 13965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:07.293766 13965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:07.293864 13965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:07.293942 13965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:07.294003 13965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:07.294064 13965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:07.303573 13965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:33267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab
--principal=oryx/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:33267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:07.304888 13965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:07.306234 13965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:07.317312 13970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:07.317342 13971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:07.317284 13973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:07.318323 13965 server_base.cc:1034] running on GCE node
I20260929 02:19:07.319075 13965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:07.320489 13965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:07.321681 13965 hybrid_clock.cc:648] HybridClock initialized: now 1790648347321626 us; error 48 us; skew 500 ppm
I20260929 02:19:07.321990 13965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:07 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648347, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:07.324915 13965 init.cc:377] Logged in from keytab as oryx/127.7.170.62@KRBTEST.COM (short username oryx)
I20260929 02:19:07.326388 13965 webserver.cc:460] Webserver started at http://127.7.170.62:46225/ using document root <none> and password file <none>
I20260929 02:19:07.327044 13965 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:07.327155 13965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:07.327432 13965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:07.330371 13965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "1c5774655bf943fdb21b7eac2c55d9b5"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "bd3379ef450004525e6198654c51b08f"
server_key_iv: "54e884a34c97657cae3fbdcf95aec476"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.331079 13965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "1c5774655bf943fdb21b7eac2c55d9b5"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "bd3379ef450004525e6198654c51b08f"
server_key_iv: "54e884a34c97657cae3fbdcf95aec476"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.336011 13965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260929 02:19:07.339238 13980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:07.340500 13965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:07.340652 13965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "1c5774655bf943fdb21b7eac2c55d9b5"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "bd3379ef450004525e6198654c51b08f"
server_key_iv: "54e884a34c97657cae3fbdcf95aec476"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.340819 13965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:07.355630 13965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:07.359623 13965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:07.359869 13965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:07.376106 13965 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:33267
I20260929 02:19:07.376139 14031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:33267 every 8 connection(s)
I20260929 02:19:07.377449 13965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:07.380997 14032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:07.386224 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 13965
I20260929 02:19:07.386401 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:07.386992 7848 external_mini_cluster.cc:1428] Setting key 971953c56f2a2e78744bb24f667b9aa5
I20260929 02:19:07.391176 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Bootstrap starting.
I20260929 02:19:07.394587 14032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:07.395696 14032 log.cc:824] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:07.398682 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: No bootstrap required, opened a new log
Sep 29 02:19:07 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:07.407053 14032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } }
I20260929 02:19:07.407512 14032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:07.407616 14032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5774655bf943fdb21b7eac2c55d9b5, State: Initialized, Role: FOLLOWER
I20260929 02:19:07.408211 14032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } }
I20260929 02:19:07.408416 14032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:07.408538 14032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:07.408694 14032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:07.409104 14035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:07.388815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32808 (local address 127.7.170.62:33267)
0929 02:19:07.389570 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:07.389581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:07.389620 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:19:07.391214 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:07.391237 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:07.391640 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:07.391945 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:07.393673 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.394984 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.396665 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.397439 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.397578 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:07.401943 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:07.401953 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:07.401967 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:07.402011 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:07.404855 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:07.405644 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:07.405651 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:07.405657 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:07.405855 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:07.406293 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:07.406297 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:07.406299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:07.406831 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:07.407063 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:07.407533 (+ 470us) server_negotiation.cc:299] Negotiation successful
0929 02:19:07.407942 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":328,"threads_started":1}
I20260929 02:19:07.410936 14032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } }
I20260929 02:19:07.411381 14032 leader_election.cc:302] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c5774655bf943fdb21b7eac2c55d9b5; no voters:
I20260929 02:19:07.411962 14032 leader_election.cc:288] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:07.412039 14037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:07.412602 14037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1c5774655bf943fdb21b7eac2c55d9b5, State: Running, Role: LEADER
I20260929 02:19:07.413143 14037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } }
I20260929 02:19:07.413926 14032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:07.415680 14038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } }
I20260929 02:19:07.415653 14039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c5774655bf943fdb21b7eac2c55d9b5. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } }
I20260929 02:19:07.416038 14038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:07.416138 14039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:07.418133 14042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:07.422837 14042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:07.431951 14042 catalog_manager.cc:1347] Generated new cluster ID: d97c463e1c9f4a388e1ef94280449af8
I20260929 02:19:07.432060 14042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:07.446524 14042 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:07.448354 14042 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:07.458124 14042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Generated new TSK 0
I20260929 02:19:07.459143 14042 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
I20260929 02:19:07.514374 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:07.827011 14058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:07.827268 14058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:07.827358 14058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:07.834407 14058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:07.834509 14058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:07.834617 14058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:07.844003 14058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:07.845340 14058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:07.846585 14058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:07.858848 14063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:07.858879 14066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:07.858811 14064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:07.859709 14058 server_base.cc:1034] running on GCE node
I20260929 02:19:07.860236 14058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:07.861115 14058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:07.862325 14058 hybrid_clock.cc:648] HybridClock initialized: now 1790648347862262 us; error 52 us; skew 500 ppm
I20260929 02:19:07.862645 14058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:07 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648347, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:07.865597 14058 init.cc:377] Logged in from keytab as oryx/127.7.170.1@KRBTEST.COM (short username oryx)
I20260929 02:19:07.867003 14058 webserver.cc:460] Webserver started at http://127.7.170.1:36533/ using document root <none> and password file <none>
I20260929 02:19:07.867654 14058 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:07.867765 14058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:07.868044 14058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:07.870951 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "2bb43afba7a493e4107301959e73d8ac"
server_key_iv: "65d0d96ec293e0cf5835e6e88d3e14bd"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.871665 14058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "2bb43afba7a493e4107301959e73d8ac"
server_key_iv: "65d0d96ec293e0cf5835e6e88d3e14bd"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.876552 14058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:19:07.879763 14073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:07.881057 14058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:19:07.881202 14058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "2bb43afba7a493e4107301959e73d8ac"
server_key_iv: "65d0d96ec293e0cf5835e6e88d3e14bd"
server_key_version: "encryptionkey@0"
I20260929 02:19:07.881371 14058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:07.924912 14058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:07.928619 14058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:07.928874 14058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:07.929774 14058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:07.931052 14058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:07.931126 14058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:07.931238 14058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:07.931288 14058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:07.959949 14058 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:43815
I20260929 02:19:07.960002 14185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:43815 every 8 connection(s)
I20260929 02:19:07.961213 14058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:07.963740 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14058
I20260929 02:19:07.963928 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:07.964542 7848 external_mini_cluster.cc:1428] Setting key 019e10d18d8eb9ce3a592bbfb459f286
Sep 29 02:19:07 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648347, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:07.979888 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:07.963606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:41657 (local address 127.7.170.62:33267)
0929 02:19:07.964071 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:07.964077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:07.965097 (+ 1020us) server_negotiation.cc:407] Connection header received
0929 02:19:07.967071 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:07.967076 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:07.967156 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:07.967243 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:07.968663 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.969390 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.970941 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.971328 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.971433 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:07.974856 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:07.974861 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:07.974864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:07.974893 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:07.977182 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:07.977814 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:07.977819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:07.977821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:07.977928 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:07.978312 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:07.978317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:07.978321 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:07.978560 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:07.978681 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:07.979327 (+ 646us) server_negotiation.cc:299] Negotiation successful
0929 02:19:07.979533 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":204,"threads_started":1}
I20260929 02:19:07.980827 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:07.964159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:41657)
0929 02:19:07.964903 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:07.964977 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:07.966819 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:07.967477 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:07.967495 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:07.967958 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:07.968509 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:07.968527 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.969528 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.969532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:07.970804 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:07.970816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:07.971540 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:07.971548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:07.971758 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:07.972425 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:07.972446 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:07.974666 (+ 2220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:07.977301 (+ 2635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:07.977310 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:07.977326 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:07.977711 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:07.978033 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:07.978038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:07.978042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:07.978213 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:07.978674 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:07.978682 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:07.978989 (+ 307us) client_negotiation.cc:769] Sending connection context
0929 02:19:07.979221 (+ 232us) client_negotiation.cc:240] Negotiation successful
0929 02:19:07.979736 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":312,"threads_started":1}
WARNING: no policy specified for oryx/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:19:07.982340 14186 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:07.982632 14186 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:07.983305 14186 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.2@KRBTEST.COM" created.
I20260929 02:19:07.985332 13997 ts_manager.cc:194] Registered new tserver with Master: 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:07.986749 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:41657
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
I20260929 02:19:08.007601 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:08.317461 14192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:08.317727 14192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:08.317864 14192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:08.324970 14192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:08.325088 14192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:08.325241 14192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:08.334762 14192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:08.336002 14192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:08.337354 14192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:08.349570 14200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:08.349699 14198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:08.349726 14197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:08.350540 14192 server_base.cc:1034] running on GCE node
I20260929 02:19:08.351115 14192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:08.351945 14192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:08.353130 14192 hybrid_clock.cc:648] HybridClock initialized: now 1790648348353085 us; error 33 us; skew 500 ppm
I20260929 02:19:08.353425 14192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:08 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648348, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:08.356314 14192 init.cc:377] Logged in from keytab as oryx/127.7.170.2@KRBTEST.COM (short username oryx)
I20260929 02:19:08.357685 14192 webserver.cc:460] Webserver started at http://127.7.170.2:44917/ using document root <none> and password file <none>
I20260929 02:19:08.358342 14192 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:08.358453 14192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:08.358709 14192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:08.361476 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "e529507e6a5247abb705321d6a1a1700"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "dc3fd0c17da8e253428adbbe75b265e1"
server_key_iv: "1d511d5ff18aebaea376155ed08fac33"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.362190 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "e529507e6a5247abb705321d6a1a1700"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "dc3fd0c17da8e253428adbbe75b265e1"
server_key_iv: "1d511d5ff18aebaea376155ed08fac33"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.367023 14192 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:08.370160 14207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.371429 14192 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:08.371557 14192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "e529507e6a5247abb705321d6a1a1700"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "dc3fd0c17da8e253428adbbe75b265e1"
server_key_iv: "1d511d5ff18aebaea376155ed08fac33"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.371708 14192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:08.403235 14192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:08.407160 14192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:08.407383 14192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:08.408214 14192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:08.409409 14192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:08.409479 14192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.409572 14192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:08.409614 14192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.437646 14192 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:37351
I20260929 02:19:08.437695 14320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:37351 every 8 connection(s)
I20260929 02:19:08.438831 14192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:08.445530 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14192
I20260929 02:19:08.445677 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:08.446187 7848 external_mini_cluster.cc:1428] Setting key f615faeb5782c87968a0f1945f984fcb
Sep 29 02:19:08 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648348, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:08.457679 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:08.441166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:37039 (local address 127.7.170.62:33267)
0929 02:19:08.441357 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:08.441363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:08.442465 (+ 1102us) server_negotiation.cc:407] Connection header received
0929 02:19:08.444196 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:08.444200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:08.444267 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:08.444347 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:08.445951 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.446894 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.448479 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.449059 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.449179 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:08.452246 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:08.452252 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:08.452256 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:08.452291 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:08.454891 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:08.455638 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:08.455644 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:08.455647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:08.455788 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:08.456128 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:08.456133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:08.456137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:08.456431 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:08.456576 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:08.457170 (+ 594us) server_negotiation.cc:299] Negotiation successful
0929 02:19:08.457365 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:19:08.458539 14323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:08.441635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:37039)
0929 02:19:08.442307 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:08.442346 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:08.443964 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:08.444604 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:08.444612 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:08.445132 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:08.445780 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:08.445811 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.447037 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.447042 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:08.448342 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:08.448354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.449235 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.449240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:08.449427 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:08.450032 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:08.450053 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:08.452035 (+ 1982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:08.455014 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:08.455022 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:08.455038 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:08.455532 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:08.455876 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:08.455879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:08.455882 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:08.456043 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:08.456540 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:08.456545 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:08.456822 (+ 277us) client_negotiation.cc:769] Sending connection context
0929 02:19:08.457064 (+ 242us) client_negotiation.cc:240] Negotiation successful
0929 02:19:08.457418 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":254,"threads_started":1}
I20260929 02:19:08.459954 14321 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:08.460239 14321 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:08.460878 14321 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
I20260929 02:19:08.462332 13997 ts_manager.cc:194] Registered new tserver with Master: e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351)
I20260929 02:19:08.463160 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:37039
WARNING: no policy specified for oryx/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
I20260929 02:19:08.490006 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:08.800087 14326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:08.800359 14326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:08.800475 14326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:08.808276 14326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:08.808393 14326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:08.808547 14326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:08.818572 14326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:33267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:08.819881 14326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:08.821203 14326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:08.833842 14331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:08.833874 14334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:08.833801 14332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:08.834645 14326 server_base.cc:1034] running on GCE node
I20260929 02:19:08.835233 14326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:08.836160 14326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:08.837369 14326 hybrid_clock.cc:648] HybridClock initialized: now 1790648348837320 us; error 42 us; skew 500 ppm
I20260929 02:19:08.837663 14326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:08 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648348, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:08.840587 14326 init.cc:377] Logged in from keytab as oryx/127.7.170.3@KRBTEST.COM (short username oryx)
I20260929 02:19:08.841981 14326 webserver.cc:460] Webserver started at http://127.7.170.3:46189/ using document root <none> and password file <none>
I20260929 02:19:08.842631 14326 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:08.842725 14326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:08.842967 14326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:08.845978 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "e4425660aad44c05acce0849cde31188"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "3d3c312675056c0097739e1d98c05134"
server_key_iv: "411903d3dce07bf04142e107b89dd0cf"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.846683 14326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "e4425660aad44c05acce0849cde31188"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "3d3c312675056c0097739e1d98c05134"
server_key_iv: "411903d3dce07bf04142e107b89dd0cf"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.851667 14326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:19:08.854990 14341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.856269 14326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:08.856401 14326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e4425660aad44c05acce0849cde31188"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "3d3c312675056c0097739e1d98c05134"
server_key_iv: "411903d3dce07bf04142e107b89dd0cf"
server_key_version: "encryptionkey@0"
I20260929 02:19:08.856591 14326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:08.872066 14326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:08.876053 14326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:08.876292 14326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:08.877156 14326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:08.878338 14326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:08.878404 14326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.878499 14326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:08.878540 14326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:08.905668 14326 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36599
I20260929 02:19:08.905709 14454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36599 every 8 connection(s)
I20260929 02:19:08.906908 14326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:08.907593 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14326
I20260929 02:19:08.907758 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:08.908269 7848 external_mini_cluster.cc:1428] Setting key 17161b0c5f2f462abd59b437b2ea7b1e
Sep 29 02:19:08 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648348, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:08.925844 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:08.909515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:54317 (local address 127.7.170.62:33267)
0929 02:19:08.909696 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:08.909702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:08.910996 (+ 1294us) server_negotiation.cc:407] Connection header received
0929 02:19:08.912734 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:08.912737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:08.912801 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:08.912875 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:08.914362 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.915209 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.916685 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.917111 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.917197 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:08.920277 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:08.920282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:08.920285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:08.920315 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:08.923261 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:08.924009 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:08.924012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:08.924014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:08.924123 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:08.924449 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:08.924451 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:08.924453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:08.924702 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:08.924802 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:08.925431 (+ 629us) server_negotiation.cc:299] Negotiation successful
0929 02:19:08.925590 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:19:08.926785 14457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:08.910073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:54317)
0929 02:19:08.910840 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:08.910892 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:08.912507 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:08.913105 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:08.913114 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:08.913632 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:08.914228 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:08.914246 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.915351 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.915355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:08.916579 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:08.916591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:08.917303 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:08.917309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:08.917511 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:08.918075 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:08.918096 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:08.920086 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:08.923508 (+ 3422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:08.923515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:08.923529 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:08.923917 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:08.924201 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:08.924204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:08.924206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:08.924376 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:08.924789 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:08.924795 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:08.925129 (+ 334us) client_negotiation.cc:769] Sending connection context
0929 02:19:08.925348 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:19:08.925717 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":310,"threads_started":1}
I20260929 02:19:08.928061 14455 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:08.928301 14455 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:08.928972 14455 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
I20260929 02:19:08.930289 13997 ts_manager.cc:194] Registered new tserver with Master: e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:08.931048 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:54317
I20260929 02:19:08.935308 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.170.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.61.keytab.
Entry for principal oryx/127.7.170.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.61.keytab.
I20260929 02:19:08.979367 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:41339
--webserver_interface=127.7.170.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.61.keytab
--principal=oryx/127.7.170.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:33267,127.7.170.61:41339
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:19:08.989508 14186 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
W20260929 02:19:09.286116 14460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:09.286388 14460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:09.286518 14460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:09.293481 14460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:09.293586 14460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:09.293663 14460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:09.293771 14460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:09.293835 14460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:09.303864 14460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:33267,127.7.170.61:41339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.61.keytab
--principal=oryx/127.7.170.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:41339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.170.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
W20260929 02:19:09.304231 14460 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.170.62:33267,127.7.170.61:41339'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260929 02:19:09.305275 14460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:09.306543 14460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:09.317687 14465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.317690 14468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.317636 14466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:09.318488 14460 server_base.cc:1034] running on GCE node
I20260929 02:19:09.319216 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:09.320567 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:09.321771 14460 hybrid_clock.cc:648] HybridClock initialized: now 1790648349321697 us; error 51 us; skew 500 ppm
I20260929 02:19:09.322111 14460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:09.329885 14460 init.cc:377] Logged in from keytab as oryx/127.7.170.61@KRBTEST.COM (short username oryx)
I20260929 02:19:09.331259 14460 webserver.cc:460] Webserver started at http://127.7.170.61:41865/ using document root <none> and password file <none>
I20260929 02:19:09.331899 14460 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:09.332000 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:09.332273 14460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:09.335196 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/instance:
uuid: "361a34c5b77b4663aa883c294a36524e"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "7d727e09c144f1c7700ad079f5e82c5a"
server_key_iv: "3effefef69f7bda5766bc8051ecec14d"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.336050 14460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal/instance:
uuid: "361a34c5b77b4663aa883c294a36524e"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "7d727e09c144f1c7700ad079f5e82c5a"
server_key_iv: "3effefef69f7bda5766bc8051ecec14d"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.340849 14460 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260929 02:19:09.344007 14475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:09.345474 14460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:09.345616 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "361a34c5b77b4663aa883c294a36524e"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "7d727e09c144f1c7700ad079f5e82c5a"
server_key_iv: "3effefef69f7bda5766bc8051ecec14d"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.345829 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:09.364730 14460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:09.368459 14460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:09.368685 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:09.386035 14460 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.61:41339
I20260929 02:19:09.386054 14527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.61:41339 every 8 connection(s)
I20260929 02:19:09.387557 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
I20260929 02:19:09.391099 14528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:09.397027 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14460
I20260929 02:19:09.397356 14528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.61@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:09.417375 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.400069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32816 (local address 127.7.170.62:33267)
0929 02:19:09.400241 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.400247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.400990 (+ 743us) server_negotiation.cc:407] Connection header received
0929 02:19:09.401975 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.401989 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.402062 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.402165 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.403730 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.404832 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.406853 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.407493 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.407618 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.411423 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.411429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.411432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.411465 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.414189 (+ 2724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.415021 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.415025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.415028 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.415206 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.415627 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.415630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.415632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.415936 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.416067 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.416803 (+ 736us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.417053 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:19:09.418430 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.399184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.7.170.61:41339)
0929 02:19:09.399907 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.399920 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.399967 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:19:09.401746 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.401767 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.402023 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.402212 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.403838 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.405337 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.406971 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.407786 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.407872 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.412030 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.412038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.412050 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.412086 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.415221 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.415794 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.415801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.415803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.415914 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.416251 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.416256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.416260 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.416593 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.416855 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.417196 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.417417 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":522,"thread_start_us":336,"threads_started":1}
I20260929 02:19:09.418474 14532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.400067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32816)
0929 02:19:09.400819 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.400880 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.401753 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.402236 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.402242 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.403036 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.403561 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.403585 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.405009 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.405013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.406722 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.406737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.407704 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.407709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.407869 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.408678 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.408715 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.411248 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.414304 (+ 3056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.414313 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.414325 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.414893 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.415316 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.415321 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.415325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.415517 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.416072 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.416080 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.416434 (+ 354us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.416699 (+ 265us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.417206 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":44,"spinlock_wait_cycles":15104,"thread_start_us":365,"threads_started":1}
I20260929 02:19:09.420082 14528 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.61@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:09.430891 14532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.420689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55106)
0929 02:19:09.420841 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.420858 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.420960 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.421286 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.421292 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.421604 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.421732 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.421739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.422696 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.422700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.423546 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.423556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.424170 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.424173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.424308 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.424741 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.424756 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.426565 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.429067 (+ 2502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.429072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.429075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.429426 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.429708 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.429711 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.429713 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.429832 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.430253 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.430256 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.430356 (+ 100us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.430454 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.430651 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 02:19:09.431051 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.420895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55106 (local address 127.7.170.61:41339)
0929 02:19:09.421080 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.421083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.421095 (+ 12us) server_negotiation.cc:407] Connection header received
0929 02:19:09.421125 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.421128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.421173 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.421254 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.421836 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.422572 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.423662 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.424058 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.424148 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.426708 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.426712 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.426715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.426741 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.428941 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.429517 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.429521 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.429523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.429626 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.429909 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.429912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.429914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.430169 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.430266 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.430523 (+ 257us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.430760 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":64}
I20260929 02:19:09.435428 14528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Bootstrap starting.
I20260929 02:19:09.438405 14528 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:09.439358 14528 log.cc:824] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:09.441980 14528 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: No bootstrap required, opened a new log
I20260929 02:19:09.448689 14528 raft_consensus.cc:348] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } }
I20260929 02:19:09.449011 14528 raft_consensus.cc:374] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:09.449106 14528 raft_consensus.cc:729] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 361a34c5b77b4663aa883c294a36524e, State: Initialized, Role: FOLLOWER
I20260929 02:19:09.449623 14528 consensus_queue.cc:260] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } }
I20260929 02:19:09.450403 14534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } } }
I20260929 02:19:09.450721 14534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:09.451390 14528 sys_catalog.cc:564] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: configured and running, proceeding with master startup.
W20260929 02:19:09.458887 14545 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:19:09.459055 14545 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:19:09.461355 14460 master_runner.cc:428] Detected that this master 361a34c5b77b4663aa883c294a36524e is joining an existing cluster
I20260929 02:19:09.461431 14460 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.170.61:41339
I20260929 02:19:09.465879 14321 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
I20260929 02:19:09.469261 14534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:09.469551 14534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } }
I20260929 02:19:09.470899 14534 leader_election.cc:288] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267)
I20260929 02:19:09.471750 14007 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "361a34c5b77b4663aa883c294a36524e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" is_pre_election: true
I20260929 02:19:09.473502 14476 leader_election.cc:302] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 361a34c5b77b4663aa883c294a36524e; no voters: 1c5774655bf943fdb21b7eac2c55d9b5
I20260929 02:19:09.474326 14534 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260929 02:19:09.482833 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.464789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.7.170.61:41339)
0929 02:19:09.464975 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.464992 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.468053 (+ 3061us) server_negotiation.cc:407] Connection header received
0929 02:19:09.468208 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.468214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.468281 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.468368 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.469606 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.470693 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.473036 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.474045 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.474156 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.476212 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.476217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.476222 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.476251 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.479680 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.480556 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.480561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.480564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.480711 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.481174 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.481178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.481181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.481541 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.481668 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.482231 (+ 563us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.482456 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260929 02:19:09.482829 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.464812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.7.170.62:33267)
0929 02:19:09.465019 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.465024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.465752 (+ 728us) server_negotiation.cc:407] Connection header received
0929 02:19:09.466036 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.466043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.466127 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.466196 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.467776 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.468732 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.470670 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.471257 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.471345 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.476080 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.476086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.476091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.476128 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.479640 (+ 3512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.480560 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.480566 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.480569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.480711 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.481180 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.481184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.481187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.481540 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.481666 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.482292 (+ 626us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.482493 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:19:09.483798 14551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.464819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32824)
0929 02:19:09.465644 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.465666 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.465876 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.466408 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.466413 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.467127 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.467598 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.467613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.468877 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.468882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.470514 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.470527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.472028 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.472035 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.472193 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.474714 (+ 2521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.474736 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.475853 (+ 1117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.479797 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.479801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.479805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.480433 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.480835 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.480840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.480854 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.481063 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.481731 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.481735 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.481974 (+ 239us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.483045 (+ 1071us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.483352 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":23,"spinlock_wait_cycles":34816,"thread_start_us":525,"threads_started":1}
I20260929 02:19:09.484390 14552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.465127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55108)
0929 02:19:09.467955 (+ 2828us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.467976 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.468090 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.468418 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.468423 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.469093 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.469440 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.469453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.470923 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.470928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.472880 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.472896 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.474320 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.474326 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.474503 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.475063 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.475089 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.476071 (+ 982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.479797 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.479801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.479805 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.480433 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.480835 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.480840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.480844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.481055 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.481730 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.481735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.481974 (+ 239us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.483427 (+ 1453us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.483736 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2709,"mutex_wait_us":45,"spinlock_wait_cycles":34432,"thread_start_us":2529,"threads_started":1}
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648347, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.62@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.170.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.60@KRBTEST.COM" created.
I20260929 02:19:09.504277 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.489652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55124)
0929 02:19:09.490264 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.490298 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.490504 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.490926 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.490934 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.491592 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.491784 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.491791 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.492730 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.492736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.493872 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.493884 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.494627 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.494633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.494808 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.495344 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.495360 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.498171 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.501352 (+ 3181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.501365 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.501419 (+ 54us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.501924 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.502355 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.502360 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.502363 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.502551 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.503195 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.503203 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.503386 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.503582 (+ 196us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.503900 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":258,"threads_started":1}
I20260929 02:19:09.505106 14531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.489842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.7.170.61:41339)
0929 02:19:09.490025 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.490030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.490379 (+ 349us) server_negotiation.cc:407] Connection header received
0929 02:19:09.490690 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.490694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.490774 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.490854 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.491922 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.492611 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.493999 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.494477 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.494588 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.498339 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.498344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.498347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.498375 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.501190 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.502066 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.502071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.502075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.502228 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.502674 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.502679 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.502682 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.502987 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.503118 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.504503 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.504734 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:19:09.507074 13997 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.170.61:41339: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "361a34c5b77b4663aa883c294a36524e"
member_type: NON_VOTER
last_known_addr {
host: "127.7.170.61"
port: 41339
}
attrs {
promote: true
}
}
dest_uuid: "1c5774655bf943fdb21b7eac2c55d9b5"
cas_config_opid_index: -1
I20260929 02:19:09.508119 13997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } }
I20260929 02:19:09.515743 14503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:09.515342 14558 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 361a34c5b77b4663aa883c294a36524e (127.7.170.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } }
I20260929 02:19:09.519330 14503 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260929 02:19:09.520169 14558 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.170.61:41339
I20260929 02:19:09.520613 14557 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:09.520855 14460 master.cc:537] Master@127.7.170.61:41339 shutting down...
I20260929 02:19:09.523466 14558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } }
I20260929 02:19:09.523773 14558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
W20260929 02:19:09.524354 13984 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260929 02:19:09.528026 13984 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 -> Peer 361a34c5b77b4663aa883c294a36524e (127.7.170.61:41339): Couldn't send request to peer 361a34c5b77b4663aa883c294a36524e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260929 02:19:09.531332 14460 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:09.531724 14460 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:09.531875 14460 tablet_replica.cc:330] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: stopping tablet replica
I20260929 02:19:09.537367 14460 master.cc:552] Master@127.7.170.61:41339 shutdown complete.
I20260929 02:19:09.537453 14460 master_runner.cc:305] Clearing existing system tablet
I20260929 02:19:09.537817 14460 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:09.537894 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:09.540212 14460 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.60.keytab.
Entry for principal oryx/127.7.170.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.60.keytab.
I20260929 02:19:09.541695 14562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:09.542029 14460 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:09.542315 14460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 02:19:09.542404 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "361a34c5b77b4663aa883c294a36524e"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "7d727e09c144f1c7700ad079f5e82c5a"
server_key_iv: "3effefef69f7bda5766bc8051ecec14d"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.542485 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:33613
--webserver_interface=127.7.170.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.60.keytab
--principal=oryx/127.7.170.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:19:09.547288 14460 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:09.554105 14460 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:19:09.554361 14460 log.cc:1196] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260929 02:19:09.554951 14460 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Deleting consensus metadata
I20260929 02:19:09.555294 14460 master_runner.cc:315] Copying system tablet from 127.7.170.62:33267
I20260929 02:19:09.557761 14460 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: tablet copy: Beginning tablet copy session from remote peer at address 127.7.170.62:33267
I20260929 02:19:09.578778 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.558665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.7.170.62:33267)
0929 02:19:09.558842 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.558848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.565471 (+ 6623us) server_negotiation.cc:407] Connection header received
0929 02:19:09.565642 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.565648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.565744 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.565865 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:09.566826 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.567922 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.569706 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.570328 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.570444 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.572610 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:09.572617 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:09.572622 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:09.572659 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:09.576070 (+ 3411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.576789 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.576794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.576797 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.576966 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:09.577349 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:09.577353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:09.577356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:09.577692 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:09.577835 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.578241 (+ 406us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.578458 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:19:09.578886 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.558591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32838)
0929 02:19:09.565331 (+ 6740us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.565356 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.565521 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.565908 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.565914 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.566428 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:09.566682 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.566695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.568076 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.568082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.569536 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.569551 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.570479 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.570485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.570682 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.571375 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:09.571398 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:09.572410 (+ 1012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:09.576189 (+ 3779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.576195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.576199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.576664 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.577063 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:09.577068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:09.577072 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:09.577243 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:09.577809 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:09.577815 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:09.577993 (+ 178us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.578141 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.578435 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6597,"thread_start_us":6582,"threads_started":1}
I20260929 02:19:09.580065 14017 tablet_copy_service.cc:140] P 1c5774655bf943fdb21b7eac2c55d9b5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 361a34c5b77b4663aa883c294a36524e ({username='oryx', principal='oryx/127.7.170.61@KRBTEST.COM'} at 127.0.0.1:32838)
I20260929 02:19:09.580310 14017 tablet_copy_service.cc:161] P 1c5774655bf943fdb21b7eac2c55d9b5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 361a34c5b77b4663aa883c294a36524e at {username='oryx', principal='oryx/127.7.170.61@KRBTEST.COM'} at 127.0.0.1:32838: session id = 361a34c5b77b4663aa883c294a36524e-00000000000000000000000000000000
I20260929 02:19:09.584450 14017 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:19:09.586678 14460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:19:09.593354 14460 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: tablet copy: Starting download of 0 data blocks...
I20260929 02:19:09.593648 14460 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: tablet copy: Starting download of 1 WAL segments...
I20260929 02:19:09.596271 14460 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:19:09.600832 14017 tablet_copy_service.cc:342] P 1c5774655bf943fdb21b7eac2c55d9b5: Request end of tablet copy session 361a34c5b77b4663aa883c294a36524e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.170.61@KRBTEST.COM'} at 127.0.0.1:32838
I20260929 02:19:09.601053 14017 tablet_copy_service.cc:433] P 1c5774655bf943fdb21b7eac2c55d9b5: ending tablet copy session 361a34c5b77b4663aa883c294a36524e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 361a34c5b77b4663aa883c294a36524e
I20260929 02:19:09.612387 14460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:09.614893 14570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.621623 14571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.625830 14573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:09.626099 14460 server_base.cc:1034] running on GCE node
I20260929 02:19:09.626600 14460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:09.627229 14460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:09.628508 14460 hybrid_clock.cc:648] HybridClock initialized: now 1790648349628441 us; error 34 us; skew 500 ppm
I20260929 02:19:09.628728 14460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:09.630873 14460 init.cc:377] Logged in from keytab as oryx/127.7.170.61@KRBTEST.COM (short username oryx)
I20260929 02:19:09.632079 14460 webserver.cc:460] Webserver started at http://127.7.170.61:45379/ using document root <none> and password file <none>
I20260929 02:19:09.632452 14460 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:09.632529 14460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:09.635468 14460 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260929 02:19:09.637388 14580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:09.637766 14460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260929 02:19:09.637882 14460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "361a34c5b77b4663aa883c294a36524e"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "7d727e09c144f1c7700ad079f5e82c5a"
server_key_iv: "3effefef69f7bda5766bc8051ecec14d"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.638041 14460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:09.654142 14460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:09.657845 14460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:09.681918 14460 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.61:41339
I20260929 02:19:09.681958 14632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.61:41339 every 8 connection(s)
I20260929 02:19:09.683020 14460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
I20260929 02:19:09.688609 14633 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:19:09.692428 14633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Bootstrap starting.
I20260929 02:19:09.714362 14633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:19:09.714901 14633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: Bootstrap complete.
I20260929 02:19:09.715938 14633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } }
I20260929 02:19:09.716202 14633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 361a34c5b77b4663aa883c294a36524e, State: Initialized, Role: LEARNER
I20260929 02:19:09.716499 14633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } }
I20260929 02:19:09.717358 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } }
I20260929 02:19:09.717628 14633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:09.717849 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: LEARNER
I20260929 02:19:09.732735 14647 catalog_manager.cc:1259] Loaded cluster ID: d97c463e1c9f4a388e1ef94280449af8
I20260929 02:19:09.732841 14647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: loading cluster ID for follower catalog manager: success
I20260929 02:19:09.736382 14647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: acquiring CA information for follower catalog manager: success
I20260929 02:19:09.738976 14647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:19:09.915056 14563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:09.915359 14563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:09.915500 14563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:09.923450 14563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:09.923552 14563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:09.923605 14563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:09.923678 14563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:09.923728 14563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:09.933533 14455 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
I20260929 02:19:09.934676 14563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.60.keytab
--principal=oryx/127.7.170.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:33613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.170.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:09.936033 14563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:09.937302 14563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:09.949231 14652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.949169 14653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:09.949205 14655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:09.950273 14563 server_base.cc:1034] running on GCE node
I20260929 02:19:09.950981 14563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:09.952342 14563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:09.953537 14563 hybrid_clock.cc:648] HybridClock initialized: now 1790648349953486 us; error 47 us; skew 500 ppm
I20260929 02:19:09.953857 14563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:09 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:09.956658 14563 init.cc:377] Logged in from keytab as oryx/127.7.170.60@KRBTEST.COM (short username oryx)
I20260929 02:19:09.958060 14563 webserver.cc:460] Webserver started at http://127.7.170.60:42831/ using document root <none> and password file <none>
I20260929 02:19:09.958681 14563 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:09.958778 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:09.959079 14563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:09.962801 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/instance:
uuid: "05a33d21143e4ec3aa50847b04adf037"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "5f7831ba3c90b2d5ee0fb5a5ae978099"
server_key_iv: "40f94f89e342beaaeaa972202febf860"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.963619 14563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal/instance:
uuid: "05a33d21143e4ec3aa50847b04adf037"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "5f7831ba3c90b2d5ee0fb5a5ae978099"
server_key_iv: "40f94f89e342beaaeaa972202febf860"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.968854 14563 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:19:09.972060 14662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:09.973453 14563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:19:09.973600 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "05a33d21143e4ec3aa50847b04adf037"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "5f7831ba3c90b2d5ee0fb5a5ae978099"
server_key_iv: "40f94f89e342beaaeaa972202febf860"
server_key_version: "encryptionkey@0"
I20260929 02:19:09.973784 14563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:09.990238 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.984629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55130)
0929 02:19:09.984817 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:09.984833 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:09.984962 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:09.985663 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:09.985668 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:09.985696 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:09.985872 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.985882 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.987092 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.987098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.988324 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:09.988336 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.989478 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.989484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:09.989668 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.989727 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 02:19:09.989811 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:19:09.989947 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:19:09.990283 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:09.984826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.7.170.61:41339)
0929 02:19:09.985347 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:09.985351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:09.985367 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:09.985425 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:09.985429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:09.985504 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:09.985602 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:09.986008 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.986965 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.988479 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:09.989314 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:09.989436 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:09.989658 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:09.989851 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 02:19:09.989972 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":226,"threads_started":1}
I20260929 02:19:09.991816 14607 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260929 02:19:09.992419 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c5774655bf943fdb21b7eac2c55d9b5. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } }
I20260929 02:19:09.992672 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: LEARNER
I20260929 02:19:09.995119 14563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:09.998977 14563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:09.999199 14563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:10.016788 14563 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.60:33613
I20260929 02:19:10.016816 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.60:33613 every 8 connection(s)
I20260929 02:19:10.018121 14563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
I20260929 02:19:10.021832 14715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:10.023443 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14563
I20260929 02:19:10.030119 14715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.60@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:10.047181 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.025433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.7.170.60:33613)
0929 02:19:10.026192 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.026210 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.026250 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:19:10.027824 (+ 1574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.027845 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.028234 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.028579 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.030387 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.032494 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.033933 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.034778 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.034950 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.039938 (+ 4988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.039948 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.039963 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.040018 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.043261 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.043987 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.043993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.043999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.044136 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.044477 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.044481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.044483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.045064 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.045325 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.045743 (+ 418us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.046170 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":299,"threads_started":1}
I20260929 02:19:10.048177 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.033513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.7.170.62:33267)
0929 02:19:10.033699 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.033705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.034234 (+ 529us) server_negotiation.cc:407] Connection header received
0929 02:19:10.034401 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.034406 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.034482 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.034576 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.035643 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.036332 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.038240 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.038821 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.038964 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.042954 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.042958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.042961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.043001 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.045544 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.046329 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.046333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.046335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.046449 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.046790 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.046795 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.046798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.047044 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.047163 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.047637 (+ 474us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.047849 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:19:10.048245 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.033341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32850)
0929 02:19:10.034096 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.034136 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.034285 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.034632 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.034637 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.035287 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.035522 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.035530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.036498 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.036503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.038115 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.038129 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.039068 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.039076 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.039195 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.039753 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.039779 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.042787 (+ 3008us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.045656 (+ 2869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.045671 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.045678 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.046227 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.046550 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.046555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.046559 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.046694 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.047161 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.047171 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.047352 (+ 181us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.047544 (+ 192us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.047844 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"spinlock_wait_cycles":20352,"thread_start_us":392,"threads_started":1}
I20260929 02:19:10.049679 14715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.60@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:10.060611 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.050329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55142)
0929 02:19:10.050507 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.050525 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.050614 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.050959 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.050962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.051240 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.051360 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.051366 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.052224 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.052227 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.053134 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.053151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.053733 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.053737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.053860 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.054314 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.054329 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.056076 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.058840 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.058843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.058846 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.059178 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.059464 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.059467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.059469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.059579 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.059962 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.059965 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.060082 (+ 117us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.060180 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.060402 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260929 02:19:10.060642 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.050579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55142 (local address 127.7.170.61:41339)
0929 02:19:10.050739 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.050743 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.050759 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:10.050803 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.050806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.050862 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.050936 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.051458 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.052133 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.053238 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.053635 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.053725 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.056220 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.056225 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.056228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.056257 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.058733 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.059259 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.059263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.059265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.059385 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.059649 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.059652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.059654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.059874 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.059981 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.060260 (+ 279us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.060423 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:19:10.061534 14715 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648349, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.60@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
I20260929 02:19:10.072206 14719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.062081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:44432)
0929 02:19:10.062262 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.062276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.062353 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.062561 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.062564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.062837 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.063348 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.063357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.064179 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.064182 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.065005 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.065018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.065592 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.065596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.065740 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.066159 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.066173 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.067883 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.070365 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.070369 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.070371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.070673 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.071012 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.071015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.071017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.071140 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.071548 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.071551 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.071649 (+ 98us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.071755 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.071913 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:19:10.072252 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.062145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.7.170.60:33613)
0929 02:19:10.062273 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.062277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.062313 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:19:10.062415 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.062418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.062465 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.062541 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.063451 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.064086 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.065107 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.065475 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.065569 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.068031 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.068036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.068039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.068065 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.070276 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.070757 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.070761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.070763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.070927 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.071218 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.071221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.071223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.071457 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.071563 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.071825 (+ 262us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.072016 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":39}
I20260929 02:19:10.076751 14715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Bootstrap starting.
I20260929 02:19:10.079731 14715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:10.080637 14715 log.cc:824] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:10.083725 14715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: No bootstrap required, opened a new log
I20260929 02:19:10.090538 14715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } }
I20260929 02:19:10.090818 14715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:10.090898 14715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05a33d21143e4ec3aa50847b04adf037, State: Initialized, Role: FOLLOWER
I20260929 02:19:10.091367 14715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } }
I20260929 02:19:10.092067 14721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } } }
I20260929 02:19:10.092342 14721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:10.092998 14715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:10.101758 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14058
W20260929 02:19:10.101737 14732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:19:10.101861 14732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:19:10.106544 14563 master_runner.cc:428] Detected that this master 05a33d21143e4ec3aa50847b04adf037 is joining an existing cluster
I20260929 02:19:10.106653 14563 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.170.60:33613
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.1@KRBTEST.COM
I20260929 02:19:10.122545 14734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.102825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.7.170.1:43815)
0929 02:19:10.103823 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.103830 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.103851 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:19:10.103901 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.103906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.104088 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.104228 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.105959 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.107128 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.108793 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.109377 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.109491 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.115295 (+ 5804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.115310 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.115326 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.115368 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.119013 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.119792 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.119801 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.119806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.119962 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.120399 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.120404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.120407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.120846 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.121234 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.121909 (+ 675us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.122173 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":864,"thread_start_us":727,"threads_started":1}
I20260929 02:19:10.124338 14181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:10.128611 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.111115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.7.170.62:33267)
0929 02:19:10.111271 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.111276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.112194 (+ 918us) server_negotiation.cc:407] Connection header received
0929 02:19:10.112445 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.112452 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.112542 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.112608 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.114450 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.115605 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.118051 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.118706 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.118800 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.122756 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.122762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.122767 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.122800 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.125380 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.126147 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.126153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.126157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.126321 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.126833 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.126838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.126841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.127193 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.127320 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.128054 (+ 734us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.128271 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260929 02:19:10.129583 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.110723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32860)
0929 02:19:10.111315 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.111333 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.112301 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.112808 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.112812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.113901 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.114288 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.114302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.116021 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.116026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.117895 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.117910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.119097 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.119103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.119436 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.120004 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.120027 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.122564 (+ 2537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.125483 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.125488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.125492 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.125950 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.126510 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.126515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.126518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.126704 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.127398 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.127403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.127708 (+ 305us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.128781 (+ 1073us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.129128 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":42,"thread_start_us":448,"threads_started":1}
I20260929 02:19:10.145991 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.112562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.7.170.61:41339)
0929 02:19:10.112735 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.112741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.126097 (+ 13356us) server_negotiation.cc:407] Connection header received
0929 02:19:10.126170 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.126176 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.126264 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.126351 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.127904 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.129040 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.133628 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.134348 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.134452 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.139305 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.139313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.139319 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.139377 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.143005 (+ 3628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.143906 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.143912 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.143915 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.144043 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.144425 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.144430 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.144433 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.144726 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.144854 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.145404 (+ 550us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.145627 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:19:10.146034 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.113060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:55146)
0929 02:19:10.125656 (+ 12596us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.125679 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.125829 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.126537 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.126543 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.127272 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.127716 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.127730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.129334 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.129339 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.133165 (+ 3826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.133182 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.136902 (+ 3720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.136910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.137123 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.137745 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.137768 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.139055 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.143188 (+ 4133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.143195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.143199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.143761 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.144152 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.144157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.144160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.144314 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.144905 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.144909 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.145129 (+ 220us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.145287 (+ 158us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.145598 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11311,"mutex_wait_us":53,"thread_start_us":8852,"threads_started":1}
I20260929 02:19:10.153342 14739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.122444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:44448)
0929 02:19:10.129928 (+ 7484us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.129947 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.130092 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.130457 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.130461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.130892 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.133432 (+ 2540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.133445 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.134844 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.134849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.136308 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.136322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.142372 (+ 6050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.142378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.142571 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.143831 (+ 1260us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.143965 (+ 134us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.145333 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.150493 (+ 5160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.150498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.150502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.150825 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.151263 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.151268 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.151271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.151789 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.152272 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.152276 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.152421 (+ 145us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.152534 (+ 113us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.152877 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7378,"thread_start_us":21210,"threads_started":1}
I20260929 02:19:10.153366 14718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.125664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.7.170.60:33613)
0929 02:19:10.126049 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.126054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.130018 (+ 3964us) server_negotiation.cc:407] Connection header received
0929 02:19:10.130196 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.130201 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.130259 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.130314 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.133570 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.134645 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.136438 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.137177 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.137269 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.147933 (+ 10664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.147938 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.147941 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.147967 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.150377 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.150930 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.150934 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.150936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.151162 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.151889 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.151893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.151895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.152173 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.152320 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.152647 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.153032 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":152,"server-negotiator.queue_time_us":252}
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648347, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.62@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
I20260929 02:19:10.167569 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.137660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:44450)
0929 02:19:10.137799 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.137817 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.137912 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.153757 (+ 15845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.153762 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.154221 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.154442 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.154456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.156312 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.156318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.157713 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.157731 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.158811 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.158819 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.159229 (+ 410us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.159887 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.159905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.162109 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.165322 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.165328 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.165332 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.165686 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.166044 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.166049 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.166053 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.166174 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.166664 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.166669 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.166810 (+ 141us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.166929 (+ 119us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.167205 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260929 02:19:10.167605 14743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.137972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.7.170.60:33613)
0929 02:19:10.153370 (+ 15398us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.153375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.153396 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:19:10.153482 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.153488 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.153567 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.154592 (+ 1025us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.154908 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.156142 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.157900 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.158607 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.158729 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.162305 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.162311 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.162314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.162342 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.165202 (+ 2860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.165803 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.165809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.165812 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.165941 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.166265 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.166269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.166271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.166558 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.166684 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.167037 (+ 353us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.167223 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15259,"thread_start_us":15261,"threads_started":1}
I20260929 02:19:10.169148 13997 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.170.60:33613: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "05a33d21143e4ec3aa50847b04adf037"
member_type: NON_VOTER
last_known_addr {
host: "127.7.170.60"
port: 33613
}
attrs {
promote: true
}
}
dest_uuid: "1c5774655bf943fdb21b7eac2c55d9b5"
cas_config_opid_index: 5
I20260929 02:19:10.169947 13997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } }
I20260929 02:19:10.171423 14558 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.174001 14690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:10.174597 14607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:19:10.178025 14690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260929 02:19:10.187390 14745 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260929 02:19:10.188189 14558 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.170.60:33613
I20260929 02:19:10.188827 14563 master.cc:537] Master@127.7.170.60:33613 shutting down...
I20260929 02:19:10.189306 14744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:10.193807 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.194170 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:10.209371 14607 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.216168 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: NON_VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: true } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.216503 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: LEARNER
I20260929 02:19:10.217121 14745 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: attempting to promote NON_VOTER 361a34c5b77b4663aa883c294a36524e to VOTER
I20260929 02:19:10.218080 14745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } }
I20260929 02:19:10.221185 14607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 LEARNER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260929 02:19:10.223577 14563 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:10.224083 14563 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:10.224334 14563 tablet_replica.cc:330] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: stopping tablet replica
I20260929 02:19:10.233584 14558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260929 02:19:10.235116 14554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.234619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:44464)
0929 02:19:10.234763 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.234953 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:33613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:19:10.235390 13984 consensus_peers.cc:487] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 -> Peer 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60:33613): Couldn't send request to peer 05a33d21143e4ec3aa50847b04adf037. Status: Network error: Client connection negotiation failed: client connection to 127.7.170.60:33613: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 02:19:10.236332 14744 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 361a34c5b77b4663aa883c294a36524e (127.7.170.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.237076 14607 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 361a34c5b77b4663aa883c294a36524e (127.7.170.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.240423 14558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.240700 14558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:10.241369 14636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.241605 14636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:10.242154 14563 master.cc:552] Master@127.7.170.60:33613 shutdown complete.
I20260929 02:19:10.242231 14563 master_runner.cc:305] Clearing existing system tablet
I20260929 02:19:10.242530 14563 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:10.242587 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:10.244554 14744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.244907 14744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:10.246146 14563 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260929 02:19:10.247480 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:10.247736 14563 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:10.247977 14563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 02:19:10.248068 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "05a33d21143e4ec3aa50847b04adf037"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "5f7831ba3c90b2d5ee0fb5a5ae978099"
server_key_iv: "40f94f89e342beaaeaa972202febf860"
server_key_version: "encryptionkey@0"
I20260929 02:19:10.251353 14563 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:10.255775 14563 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260929 02:19:10.255914 14563 log.cc:1196] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260929 02:19:10.256307 14563 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Deleting consensus metadata
I20260929 02:19:10.256513 14563 master_runner.cc:315] Copying system tablet from 127.7.170.62:33267
I20260929 02:19:10.258854 14563 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: tablet copy: Beginning tablet copy session from remote peer at address 127.7.170.62:33267
I20260929 02:19:10.272091 14188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.259581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.7.170.62:33267)
0929 02:19:10.259838 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.259844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.261457 (+ 1613us) server_negotiation.cc:407] Connection header received
0929 02:19:10.261631 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.261636 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.261712 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.261789 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.263100 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.263823 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.265012 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.265457 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.265569 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.267278 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.267282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.267285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.267317 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.269717 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.270283 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.270288 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.270291 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.270415 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.270754 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.270759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.270762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.271015 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.271138 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.271524 (+ 386us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.271722 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 02:19:10.272140 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.259580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:32862)
0929 02:19:10.261348 (+ 1768us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.261368 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.261509 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.261833 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.261838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.262348 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.262956 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.262970 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.263960 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.263966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.264860 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.264871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.265603 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.265612 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.265790 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.266373 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.266390 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.267091 (+ 701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.269834 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.269840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.269844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.270179 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.270517 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.270522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.270526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.270659 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.271120 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.271125 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.271286 (+ 161us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.271406 (+ 120us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.271722 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1636,"thread_start_us":1525,"threads_started":1}
I20260929 02:19:10.273058 14017 tablet_copy_service.cc:140] P 1c5774655bf943fdb21b7eac2c55d9b5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 05a33d21143e4ec3aa50847b04adf037 ({username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:32862)
I20260929 02:19:10.273216 14017 tablet_copy_service.cc:161] P 1c5774655bf943fdb21b7eac2c55d9b5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 05a33d21143e4ec3aa50847b04adf037 at {username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:32862: session id = 05a33d21143e4ec3aa50847b04adf037-00000000000000000000000000000000
I20260929 02:19:10.275803 14017 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Tablet Copy: opened 0 blocks and 1 log segments
I20260929 02:19:10.277736 14563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260929 02:19:10.278628 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14058
I20260929 02:19:10.283113 14563 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: tablet copy: Starting download of 0 data blocks...
I20260929 02:19:10.283303 14563 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: tablet copy: Starting download of 1 WAL segments...
I20260929 02:19:10.285470 14563 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260929 02:19:10.288425 14017 tablet_copy_service.cc:342] P 1c5774655bf943fdb21b7eac2c55d9b5: Request end of tablet copy session 05a33d21143e4ec3aa50847b04adf037-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:32862
I20260929 02:19:10.288543 14017 tablet_copy_service.cc:433] P 1c5774655bf943fdb21b7eac2c55d9b5: ending tablet copy session 05a33d21143e4ec3aa50847b04adf037-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 05a33d21143e4ec3aa50847b04adf037
I20260929 02:19:10.298377 14563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:19:10.299492 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:43815
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:33267
--webserver_port=36533
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:10.300832 14758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:10.306584 14759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:10.314268 14563 server_base.cc:1034] running on GCE node
W20260929 02:19:10.314332 14762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:10.314836 14563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:10.315476 14563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:10.316684 14563 hybrid_clock.cc:648] HybridClock initialized: now 1790648350316639 us; error 33 us; skew 500 ppm
I20260929 02:19:10.316917 14563 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:10.318849 14563 init.cc:377] Logged in from keytab as oryx/127.7.170.60@KRBTEST.COM (short username oryx)
I20260929 02:19:10.329275 14563 webserver.cc:460] Webserver started at http://127.7.170.60:33911/ using document root <none> and password file <none>
I20260929 02:19:10.329564 14563 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:10.329654 14563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:10.331988 14563 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:19:10.333496 14769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:10.333885 14563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260929 02:19:10.333989 14563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "05a33d21143e4ec3aa50847b04adf037"
format_stamp: "Formatted at 2026-09-29 02:19:09 on dist-test-slave-8cs2"
server_key: "5f7831ba3c90b2d5ee0fb5a5ae978099"
server_key_iv: "40f94f89e342beaaeaa972202febf860"
server_key_version: "encryptionkey@0"
I20260929 02:19:10.334159 14563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:10.370138 14563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:10.373256 14563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:10.424641 14563 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.60:33613
I20260929 02:19:10.424703 14822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.60:33613 every 8 connection(s)
I20260929 02:19:10.425892 14563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
I20260929 02:19:10.430737 14823 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:19:10.434167 14823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Bootstrap starting.
I20260929 02:19:10.449291 14823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:19:10.449689 14823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Bootstrap complete.
I20260929 02:19:10.450507 14823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } }
I20260929 02:19:10.450736 14823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 05a33d21143e4ec3aa50847b04adf037, State: Initialized, Role: LEARNER
I20260929 02:19:10.451000 14823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } }
I20260929 02:19:10.451978 14823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:10.451707 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.452278 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:19:10.459097 14837 catalog_manager.cc:1259] Loaded cluster ID: d97c463e1c9f4a388e1ef94280449af8
I20260929 02:19:10.459168 14837 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: loading cluster ID for follower catalog manager: success
I20260929 02:19:10.461627 14837 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: acquiring CA information for follower catalog manager: success
I20260929 02:19:10.463519 14837 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:19:10.708293 14761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:10.708590 14761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:10.708722 14761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:10.716975 14761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:10.717093 14761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:10.717242 14761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:10.727286 14761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:43815
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=36533
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:10.728505 14761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:10.729828 14761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:10.745047 14842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:10.745369 14843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:10.745065 14845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:10.746014 14761 server_base.cc:1034] running on GCE node
I20260929 02:19:10.746557 14761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:10.747394 14761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:10.748593 14761 hybrid_clock.cc:648] HybridClock initialized: now 1790648350748536 us; error 40 us; skew 500 ppm
I20260929 02:19:10.748945 14761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:10.751821 14761 init.cc:377] Logged in from keytab as oryx/127.7.170.1@KRBTEST.COM (short username oryx)
I20260929 02:19:10.753262 14761 webserver.cc:460] Webserver started at http://127.7.170.1:36533/ using document root <none> and password file <none>
I20260929 02:19:10.753932 14761 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:10.754055 14761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:10.759369 14761 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:19:10.762444 14852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:10.763785 14761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:19:10.763960 14761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c"
format_stamp: "Formatted at 2026-09-29 02:19:07 on dist-test-slave-8cs2"
server_key: "2bb43afba7a493e4107301959e73d8ac"
server_key_iv: "65d0d96ec293e0cf5835e6e88d3e14bd"
server_key_version: "encryptionkey@0"
I20260929 02:19:10.764851 14761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:10.779417 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.773505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.7.170.60:33613)
0929 02:19:10.773946 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.773951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.773967 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:10.774034 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.774037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.774117 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.774224 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:10.774591 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.775652 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.777775 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.778617 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.778740 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.778892 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.779075 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.779167 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":236,"threads_started":1}
I20260929 02:19:10.779457 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.773311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:44468)
0929 02:19:10.773703 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.773722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.773846 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.774244 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.774248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.774271 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:10.774472 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.774483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.775776 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.775782 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.777628 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.777642 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.778770 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.778775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.778941 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.778974 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.779058 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.779180 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":182,"threads_started":1}
I20260929 02:19:10.780226 14798 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260929 02:19:10.780735 14826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c5774655bf943fdb21b7eac2c55d9b5. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:10.781008 14826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: LEARNER
I20260929 02:19:10.788133 14761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:10.791910 14761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:10.792135 14761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:10.793535 14761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:10.794849 14761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:10.794931 14761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:10.795019 14761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:10.795073 14761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:10.829252 14761 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:43815
I20260929 02:19:10.829309 14967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:43815 every 8 connection(s)
I20260929 02:19:10.830618 14761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:10.837872 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14761
I20260929 02:19:10.838204 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14192
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:10.858183 14858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.833474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:38075 (local address 127.7.170.60:33613)
0929 02:19:10.833660 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.833666 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.837421 (+ 3755us) server_negotiation.cc:407] Connection header received
0929 02:19:10.837642 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.837647 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.837721 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.837787 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.840532 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.841789 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.844562 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.845210 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.845304 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.850142 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.850149 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.850154 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.850189 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.853161 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.854135 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.854141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.854145 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.854284 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.854818 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.854822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.854826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.855218 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.855335 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.857188 (+ 1853us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.857394 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:19:10.858191 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.833888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.7.170.1:38075)
0929 02:19:10.837327 (+ 3439us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.837344 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.837515 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.838263 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.838275 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.839373 (+ 1098us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.840323 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.840348 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.842009 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.842015 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.844360 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.844390 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.845671 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.845679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.846002 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.846928 (+ 926us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.846958 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.849919 (+ 2961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.853339 (+ 3420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.853347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.853377 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.853980 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.854421 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.854426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.854430 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.854671 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.855444 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.855455 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.855861 (+ 406us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.856186 (+ 325us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.856679 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3310,"mutex_wait_us":71,"spinlock_wait_cycles":78080,"thread_start_us":3021,"threads_started":1}
I20260929 02:19:10.860203 14968 heartbeater.cc:347] Connected to a master server at 127.7.170.60:33613
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648350, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:10.860539 14968 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:10.861590 14968 heartbeater.cc:511] Master 127.7.170.60:33613 requested a full tablet report, sending...
Sep 29 02:19:10 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.2@KRBTEST.COM
I20260929 02:19:10.864048 14787 ts_manager.cc:194] Registered new tserver with Master: 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:10.867767 14972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.833592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:58017 (local address 127.7.170.62:33267)
0929 02:19:10.834507 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.834513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.835160 (+ 647us) server_negotiation.cc:407] Connection header received
0929 02:19:10.837130 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.837136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.837221 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.837335 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.848832 (+ 11497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.850625 (+ 1793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.853250 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.853855 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.853945 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.858645 (+ 4700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.858652 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.858656 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.858696 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.862397 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.863800 (+ 1403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.863805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.863809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.863945 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.864360 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.864365 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.864369 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.864703 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.864827 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.867135 (+ 2308us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.867348 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":703,"threads_started":1}
I20260929 02:19:10.868160 14973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.833887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:58017)
0929 02:19:10.835018 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.835054 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.836820 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.844789 (+ 7969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.844794 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.845258 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.845543 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.845554 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.850783 (+ 5229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.850789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.853104 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.853115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.854248 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.854254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.854423 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.855011 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.855032 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.858402 (+ 3370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.863200 (+ 4798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.863205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.863209 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.863649 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.864078 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.864083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.864088 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.864235 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.864881 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.864886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.865049 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.865762 (+ 713us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.866054 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":909,"thread_start_us":680,"threads_started":1}
I20260929 02:19:10.869318 14969 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:10.869473 14969 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:10.869946 14969 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
I20260929 02:19:10.870041 14976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.837633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:34825 (local address 127.7.170.61:41339)
0929 02:19:10.851519 (+ 13886us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.851524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.851547 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:19:10.851603 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.851609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.851688 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.853038 (+ 1350us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.853227 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.854230 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.856054 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.857321 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.857962 (+ 641us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.861290 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.861297 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.861302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.861342 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.864480 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.865247 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.865252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.865255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.865392 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.866130 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.866135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.866139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.868745 (+ 2606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.868885 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.869431 (+ 546us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.869691 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13772,"thread_start_us":13688,"threads_started":1}
I20260929 02:19:10.870345 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.837622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.7.170.1:34825)
0929 02:19:10.848529 (+ 10907us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:10.848547 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:10.848654 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:10.851841 (+ 3187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:10.851846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:10.852541 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:10.852859 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.852873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.854372 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.854377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.855875 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:10.855894 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.857522 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.857529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:10.857700 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.858440 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:10.858460 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:10.860967 (+ 2507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:10.864619 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.864624 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.864627 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.865101 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.865492 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:10.865496 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:10.865500 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:10.866000 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:10.868872 (+ 2872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:10.868877 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:10.869084 (+ 207us) client_negotiation.cc:769] Sending connection context
0929 02:19:10.869320 (+ 236us) client_negotiation.cc:240] Negotiation successful
0929 02:19:10.869691 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10776,"spinlock_wait_cycles":723712,"thread_start_us":8872,"threads_started":1}
I20260929 02:19:10.871071 13997 ts_manager.cc:194] Re-registered known tserver with Master: 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:10.871330 14970 heartbeater.cc:347] Connected to a master server at 127.7.170.61:41339
I20260929 02:19:10.871460 14970 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:10.871853 14970 heartbeater.cc:511] Master 127.7.170.61:41339 requested a full tablet report, sending...
I20260929 02:19:10.871994 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:58017
I20260929 02:19:10.873474 14597 ts_manager.cc:194] Registered new tserver with Master: 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:10.876055 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:10.839409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.7.170.2:37351)
0929 02:19:10.850509 (+ 11100us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:10.850517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:10.850545 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:19:10.850606 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:10.850613 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:10.850876 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:10.851047 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:10.853129 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.854671 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.856606 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:10.857449 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:10.857577 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:10.864870 (+ 7293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:10.864886 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:10.864902 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:10.864983 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:10.872650 (+ 7667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.873983 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.873993 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.873996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.874124 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:10.874502 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:10.874506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:10.874508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:10.874879 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:10.875147 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:10.875470 (+ 323us) server_negotiation.cc:299] Negotiation successful
0929 02:19:10.875718 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10940,"thread_start_us":11313,"threads_started":1}
I20260929 02:19:10.877267 14316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:10.958189 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14192
I20260929 02:19:10.971091 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:37351
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:33267
--webserver_port=44917
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:11.286463 14980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:11.286734 14980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:11.286837 14980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:11.294003 14980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:11.294116 14980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:11.294237 14980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:11.304374 14980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:37351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=44917
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:11.305732 14980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:11.307085 14980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:11.320199 14985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:11.320303 14988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:11.320240 14986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:11.320991 14980 server_base.cc:1034] running on GCE node
I20260929 02:19:11.321607 14980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:11.322546 14980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:11.323741 14980 hybrid_clock.cc:648] HybridClock initialized: now 1790648351323681 us; error 57 us; skew 500 ppm
I20260929 02:19:11.324048 14980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:11.327005 14980 init.cc:377] Logged in from keytab as oryx/127.7.170.2@KRBTEST.COM (short username oryx)
I20260929 02:19:11.328392 14980 webserver.cc:460] Webserver started at http://127.7.170.2:44917/ using document root <none> and password file <none>
I20260929 02:19:11.329167 14980 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:11.329301 14980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:11.334862 14980 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:11.337662 14995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.338079 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: NON_VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: true } } }
I20260929 02:19:11.338402 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:11.339430 14980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:11.339807 14745 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: attempting to promote NON_VOTER 05a33d21143e4ec3aa50847b04adf037 to VOTER
I20260929 02:19:11.340476 14745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } }
I20260929 02:19:11.339589 14980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "e529507e6a5247abb705321d6a1a1700"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "dc3fd0c17da8e253428adbbe75b265e1"
server_key_iv: "1d511d5ff18aebaea376155ed08fac33"
server_key_version: "encryptionkey@0"
I20260929 02:19:11.340869 14980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:11.342592 14798 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 LEARNER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:19:11.342638 14607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Refusing update from remote peer 1c5774655bf943fdb21b7eac2c55d9b5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260929 02:19:11.343083 14996 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:19:11.343453 14745 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260929 02:19:11.346391 14996 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.347038 14607 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.348088 14798 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.350095 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.350348 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:11.350570 14745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.350718 14745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:11.351277 15003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.351673 15003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:11.350131 14996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.353216 14996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:11.353569 15004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:11.353924 15004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:11.378633 14980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:11.382938 14980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:11.383193 14980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:11.384124 14980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:11.385485 14980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:11.385579 14980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.385677 14980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:11.385741 14980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.414906 14980 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:37351
I20260929 02:19:11.414928 15116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:37351 every 8 connection(s)
I20260929 02:19:11.416340 14980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:11.420197 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 14980
I20260929 02:19:11.420488 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14326
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:11.446511 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.419384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:51955 (local address 127.7.170.61:41339)
0929 02:19:11.420066 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.420071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.421538 (+ 1467us) server_negotiation.cc:407] Connection header received
0929 02:19:11.424495 (+ 2957us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.424502 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.424596 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.424678 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.427204 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.428465 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.431269 (+ 2804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.432107 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.432227 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.438693 (+ 6466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:11.438700 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:11.438704 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:11.438754 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:11.442171 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.443262 (+ 1091us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.443269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.443273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.443432 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.444033 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.444037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.444041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.444406 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:11.444541 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:11.445755 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0929 02:19:11.446037 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":437,"threads_started":1}
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:11.448849 15124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.419672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.7.170.2:51955)
0929 02:19:11.421434 (+ 1762us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.421449 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.424139 (+ 2690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.425262 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.425275 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.426126 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.426990 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.427017 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.428694 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.428700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.431089 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.431105 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.432661 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.432669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.433032 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.434220 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:11.434242 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:11.438313 (+ 4071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:11.442334 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.442342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.442378 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.443106 (+ 728us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.443640 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.443645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.443649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.443916 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.444687 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:11.444698 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:11.445240 (+ 542us) client_negotiation.cc:769] Sending connection context
0929 02:19:11.446354 (+ 1114us) client_negotiation.cc:240] Negotiation successful
0929 02:19:11.446926 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1627,"mutex_wait_us":249,"thread_start_us":1210,"threads_started":1}
I20260929 02:19:11.451148 15119 heartbeater.cc:347] Connected to a master server at 127.7.170.61:41339
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.3@KRBTEST.COM
I20260929 02:19:11.451539 15119 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:11.452355 15119 heartbeater.cc:511] Master 127.7.170.61:41339 requested a full tablet report, sending...
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
I20260929 02:19:11.454161 14597 ts_manager.cc:194] Registered new tserver with Master: e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351)
I20260929 02:19:11.456713 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.419067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56541 (local address 127.7.170.62:33267)
0929 02:19:11.419547 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.419553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.420865 (+ 1312us) server_negotiation.cc:407] Connection header received
0929 02:19:11.436241 (+ 15376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.436247 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.436323 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.436391 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.439120 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.440871 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.442742 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.443387 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.443476 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.447630 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:11.447637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:11.447642 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:11.447692 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:11.451482 (+ 3790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.452449 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.452455 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.452458 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.452616 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.453071 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.453076 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.453079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.453400 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:11.453808 (+ 408us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:11.455686 (+ 1878us) server_negotiation.cc:299] Negotiation successful
0929 02:19:11.455891 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":421,"threads_started":1}
I20260929 02:19:11.457288 15123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.419672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:56541)
0929 02:19:11.420698 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.420756 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.436075 (+ 15319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.436605 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.436609 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.438566 (+ 1957us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.438894 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.438909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.441063 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.441069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.442586 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.442600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.443818 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.443824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.443999 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.444577 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:11.444598 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:11.447452 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:11.451662 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.451668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.451671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.452295 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.452741 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.452746 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.452750 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.452917 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.453882 (+ 965us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:11.453888 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:11.454037 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 02:19:11.454173 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 02:19:11.454430 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":429,"threads_started":1}
I20260929 02:19:11.458134 15118 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:11.458271 15118 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:11.458796 15118 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
I20260929 02:19:11.459549 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.421445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.7.170.3:36599)
0929 02:19:11.433867 (+ 12422us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.433873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.437822 (+ 3949us) server_negotiation.cc:407] Connection header received
0929 02:19:11.437948 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.437957 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.438237 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.438389 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.440619 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.442222 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.444023 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.444844 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.444973 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.452705 (+ 7732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:11.452723 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:11.452739 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:11.452803 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:11.456379 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.457218 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.457226 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.457230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.457390 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.457807 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.457820 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.457824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.458242 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:11.458516 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:11.458881 (+ 365us) server_negotiation.cc:299] Negotiation successful
0929 02:19:11.459175 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12266,"thread_start_us":12122,"threads_started":1}
I20260929 02:19:11.459801 13997 ts_manager.cc:194] Re-registered known tserver with Master: e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351)
I20260929 02:19:11.460865 14450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:11.460829 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:56541
I20260929 02:19:11.461675 15128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.422029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:58975 (local address 127.7.170.60:33613)
0929 02:19:11.435662 (+ 13633us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.435669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.436308 (+ 639us) server_negotiation.cc:407] Connection header received
0929 02:19:11.436383 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.436387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.436461 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.436554 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.444652 (+ 8098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.445927 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.448063 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.448659 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.448771 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.454631 (+ 5860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:11.454640 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:11.454644 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:11.454686 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:11.457924 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.458701 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.458706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.458710 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.458930 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:11.459346 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:11.459350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:11.459354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:11.460477 (+ 1123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:11.460625 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:11.461037 (+ 412us) server_negotiation.cc:299] Negotiation successful
0929 02:19:11.461248 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13505,"thread_start_us":13369,"threads_started":1}
I20260929 02:19:11.461791 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.421624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.7.170.2:58975)
0929 02:19:11.436070 (+ 14446us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.436088 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.436204 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.436754 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.436760 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.444151 (+ 7391us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.444462 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.444475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.446105 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.446110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.447877 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.447891 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.449044 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.449051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:11.449217 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:11.450082 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:11.450103 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:11.454408 (+ 4305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:11.458059 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.458063 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.458066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.458565 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.459061 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:11.459066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:11.459069 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:11.459227 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:11.460619 (+ 1392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:11.460624 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:11.460772 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 02:19:11.460881 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 02:19:11.461407 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14331,"thread_start_us":14191,"threads_started":1}
I20260929 02:19:11.462484 15117 heartbeater.cc:347] Connected to a master server at 127.7.170.60:33613
I20260929 02:19:11.462598 15117 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:11.462924 15117 heartbeater.cc:511] Master 127.7.170.60:33613 requested a full tablet report, sending...
I20260929 02:19:11.464005 14787 ts_manager.cc:194] Registered new tserver with Master: e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351)
I20260929 02:19:11.542516 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14326
I20260929 02:19:11.555756 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:36599
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:33267
--webserver_port=46189
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:11.873363 15130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:11.873646 15130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:11.873764 15130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260929 02:19:11.875937 14969 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
W20260929 02:19:11.881865 15130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:11.881974 15130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:11.882093 15130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:11.891947 15130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:36599
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=46189
--tserver_master_addrs=127.7.170.62:33267,127.7.170.61:41339,127.7.170.60:33613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:11.893299 15130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:11.894579 15130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:11.907366 15138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:11.907402 15136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:11.907428 15135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:11.908201 15130 server_base.cc:1034] running on GCE node
I20260929 02:19:11.908777 15130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:11.909775 15130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:11.910996 15130 hybrid_clock.cc:648] HybridClock initialized: now 1790648351910912 us; error 48 us; skew 500 ppm
I20260929 02:19:11.911288 15130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:11 dist-test-slave-8cs2 krb5kdc[13952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:11.914464 15130 init.cc:377] Logged in from keytab as oryx/127.7.170.3@KRBTEST.COM (short username oryx)
I20260929 02:19:11.916021 15130 webserver.cc:460] Webserver started at http://127.7.170.3:46189/ using document root <none> and password file <none>
I20260929 02:19:11.916682 15130 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:11.916795 15130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:11.922300 15130 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:19:11.925050 15145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.926409 15130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:19:11.926555 15130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e4425660aad44c05acce0849cde31188"
format_stamp: "Formatted at 2026-09-29 02:19:08 on dist-test-slave-8cs2"
server_key: "3d3c312675056c0097739e1d98c05134"
server_key_iv: "411903d3dce07bf04142e107b89dd0cf"
server_key_version: "encryptionkey@0"
I20260929 02:19:11.927469 15130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:11.949358 15130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:11.953233 15130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:11.953474 15130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:11.954449 15130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:11.955830 15130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:11.955924 15130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.956028 15130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:11.956095 15130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:11.985607 15130 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36599
I20260929 02:19:11.985674 15258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36599 every 8 connection(s)
I20260929 02:19:11.987022 15130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:11.994465 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 15130
Sep 29 02:19:12 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
Sep 29 02:19:12 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.60@KRBTEST.COM
Sep 29 02:19:12 dist-test-slave-8cs2 krb5kdc[13952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648351, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.61@KRBTEST.COM
I20260929 02:19:12.014920 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.992266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35585 (local address 127.7.170.60:33613)
0929 02:19:11.992792 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.992798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.992816 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:19:11.994192 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.994198 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.994283 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.994480 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.996712 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.997950 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.000759 (+ 2809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.001423 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.001525 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.007076 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.007084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.007089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.007139 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.012142 (+ 5003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012812 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.012817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.012821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.012996 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.013350 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.013354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.013357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.013692 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.013815 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.014365 (+ 550us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.014570 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":307,"threads_started":1}
I20260929 02:19:12.015026 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.989814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:51091 (local address 127.7.170.61:41339)
0929 02:19:11.990231 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.990237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.991633 (+ 1396us) server_negotiation.cc:407] Connection header received
0929 02:19:11.993908 (+ 2275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.993913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.994037 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.994138 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.996706 (+ 2568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.997813 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.001279 (+ 3466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.001888 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.002001 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.007768 (+ 5767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.007775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.007780 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.007833 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.011169 (+ 3336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012022 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.012028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.012032 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.012204 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012827 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.012833 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.012837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.013240 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.013370 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.014398 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.014629 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":634,"threads_started":1}
I20260929 02:19:12.016614 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.989805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:53835 (local address 127.7.170.62:33267)
0929 02:19:11.990535 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:11.990540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:11.991585 (+ 1045us) server_negotiation.cc:407] Connection header received
0929 02:19:11.993804 (+ 2219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:11.993810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:11.993905 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:11.993995 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:11.997038 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.998075 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.000740 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.001405 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.001507 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.006622 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.006629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.006635 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.006678 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.010074 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012288 (+ 2214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.012293 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.012297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.012438 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.013370 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.013375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.013377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.013689 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.013812 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.016029 (+ 2217us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.016258 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":522,"threads_started":1}
I20260929 02:19:12.016762 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.990417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:53835)
0929 02:19:11.991388 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.991457 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.993531 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.994256 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.994267 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.995281 (+ 1014us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.996694 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.996703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.998237 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.998242 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.000567 (+ 2325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.000582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.001873 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.001878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.002290 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.003114 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:12.003146 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:12.006323 (+ 3177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:12.010207 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.010216 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.010236 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.010852 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.012817 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012822 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.012825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.013049 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.015658 (+ 2609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:12.015661 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:12.015784 (+ 123us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.015885 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.016151 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":526,"spinlock_wait_cycles":83200,"thread_start_us":421,"threads_started":1}
I20260929 02:19:12.017099 15267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.992101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.7.170.3:35585)
0929 02:19:11.992639 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.992655 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.993535 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.995042 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.995046 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.995641 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.996511 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.996539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.998148 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.998154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.000567 (+ 2413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.000582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.001799 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.001806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.002290 (+ 484us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.003114 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:12.003156 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:12.006741 (+ 3585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:12.012249 (+ 5508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012254 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.012258 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.012702 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.013122 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.013127 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.013130 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.013253 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.013882 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:12.013886 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:12.013987 (+ 101us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.014280 (+ 293us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.014822 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"mutex_wait_us":234,"spinlock_wait_cycles":12544,"thread_start_us":257,"threads_started":1}
I20260929 02:19:12.017647 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:11.990417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.7.170.3:51091)
0929 02:19:11.991388 (+ 971us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:11.991460 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:11.993534 (+ 2074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:11.994305 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:11.994309 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:11.995281 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:11.996511 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:11.996539 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:11.997997 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:11.998002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.001101 (+ 3099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.001122 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.002310 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.002317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.002527 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.003255 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:12.003279 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:12.007528 (+ 4249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:12.011305 (+ 3777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.011310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.011314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.011897 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.012335 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:12.012340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:12.012345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:12.012690 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:12.013461 (+ 771us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:12.013470 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:12.013949 (+ 479us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.015271 (+ 1322us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.015571 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":1269,"spinlock_wait_cycles":33920,"thread_start_us":367,"threads_started":1}
I20260929 02:19:12.018520 15259 heartbeater.cc:347] Connected to a master server at 127.7.170.60:33613
I20260929 02:19:12.018842 15259 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:12.019090 15261 heartbeater.cc:347] Connected to a master server at 127.7.170.61:41339
I20260929 02:19:12.019227 15261 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:12.019826 15259 heartbeater.cc:511] Master 127.7.170.60:33613 requested a full tablet report, sending...
I20260929 02:19:12.019851 15261 heartbeater.cc:511] Master 127.7.170.61:41339 requested a full tablet report, sending...
I20260929 02:19:12.020583 15260 heartbeater.cc:347] Connected to a master server at 127.7.170.62:33267
I20260929 02:19:12.020699 15260 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:12.021170 15260 heartbeater.cc:511] Master 127.7.170.62:33267 requested a full tablet report, sending...
I20260929 02:19:12.021793 14787 ts_manager.cc:194] Registered new tserver with Master: e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:12.022047 13997 ts_manager.cc:194] Re-registered known tserver with Master: e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:12.022799 14597 ts_manager.cc:194] Registered new tserver with Master: e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:12.023094 13997 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:53835
I20260929 02:19:12.040809 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.017687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.7.170.61:41339)
0929 02:19:12.017863 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.017868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.023651 (+ 5783us) server_negotiation.cc:407] Connection header received
0929 02:19:12.023799 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.023804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.023881 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.023970 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.026908 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.027879 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.029902 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.030509 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.030639 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.035161 (+ 4522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.035168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.035173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.035212 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.037710 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.038840 (+ 1130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.038844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.038847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.038988 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.039401 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.039405 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.039408 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.039730 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.039982 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.040269 (+ 287us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.040482 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:19:12.041620 15264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.001987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45854 (local address 127.7.170.62:33267)
0929 02:19:12.018366 (+ 16379us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.018370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.020173 (+ 1803us) server_negotiation.cc:407] Connection header received
0929 02:19:12.020266 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.020272 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.020348 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.020436 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.026543 (+ 6107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.027524 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.029451 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.030051 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.030153 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.033853 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.033859 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.033864 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.033907 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.038053 (+ 4146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.038785 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.038791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.038794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.038967 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.039379 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.039385 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.039388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.039770 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.039999 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.041033 (+ 1034us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.041262 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16288,"thread_start_us":27192,"threads_started":1}
I20260929 02:19:12.044181 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.020679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.7.170.60:33613)
0929 02:19:12.021670 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.021675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.029836 (+ 8161us) server_negotiation.cc:407] Connection header received
0929 02:19:12.030009 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.030015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.030102 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.030194 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.032514 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.033244 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.034571 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.035089 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.035204 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.037933 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.037940 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.037945 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.037995 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.040990 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.041925 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.041930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.041933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.042086 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.042673 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.042676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.042678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.042942 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.043114 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.043654 (+ 540us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.043889 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":883}
I20260929 02:19:12.046552 13997 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:12.053349 13997 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:12.087883 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.070761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37351 (local address 127.0.0.1:49084)
0929 02:19:12.071277 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:12.071298 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:12.071414 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:12.078214 (+ 6800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:12.078221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:12.078256 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:12.078792 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.078812 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.081936 (+ 3124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.081941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.085035 (+ 3094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.085052 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.087100 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.087106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.087240 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.087264 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.087341 (+ 77us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.087480 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":811,"threads_started":1}
I20260929 02:19:12.089371 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.070953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49084 (local address 127.7.170.2:37351)
0929 02:19:12.071404 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.071412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.071443 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:19:12.071541 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.071549 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.071834 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.079097 (+ 7263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:12.079327 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.080913 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.085236 (+ 4323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.086894 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.087034 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.087267 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.087419 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.089107 (+ 1688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":216,"threads_started":1}
I20260929 02:19:12.093704 15288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.071878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36599 (local address 127.0.0.1:57646)
0929 02:19:12.082171 (+ 10293us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:12.082196 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:12.082311 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:12.087337 (+ 5026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:12.087341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:12.087367 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:12.087720 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.087732 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.089473 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.089479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.091136 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.091147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.093010 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.093016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.093158 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.093181 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.093260 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.093378 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10147,"thread_start_us":14732,"threads_started":1}
I20260929 02:19:12.093801 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.072240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.7.170.3:36599)
0929 02:19:12.085472 (+ 13232us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.085492 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.085533 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:19:12.085643 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.085654 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.085984 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.086208 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:12.087922 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.089346 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.091282 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.092308 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.092417 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.092631 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.093321 (+ 690us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.093492 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12321,"spinlock_wait_cycles":204672,"thread_start_us":12724,"threads_started":1}
I20260929 02:19:12.093547 15051 tablet_service.cc:1467] Processing CreateTablet for tablet e46a760d7ab54f519a56e8d9cb046bef (DEFAULT_TABLE table=test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:12.094975 15287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.072071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:43815 (local address 127.0.0.1:57628)
0929 02:19:12.086839 (+ 14768us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:12.086854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:12.086993 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:12.087903 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:12.087908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:12.087932 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:12.088339 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.088348 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.090083 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.090088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.092067 (+ 1979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.092080 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.094272 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.094313 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.094483 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.094518 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.094610 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.094731 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14658,"thread_start_us":10324,"threads_started":1}
I20260929 02:19:12.095186 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.072274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57628 (local address 127.7.170.1:43815)
0929 02:19:12.087115 (+ 14841us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.087124 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.087152 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:19:12.087243 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.087251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.087506 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.088493 (+ 987us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:12.088652 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.089947 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.092236 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.094078 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.094260 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.094456 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.094649 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.094811 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14662,"thread_start_us":15375,"threads_started":1}
I20260929 02:19:12.098124 15051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46a760d7ab54f519a56e8d9cb046bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:12.097190 15193 tablet_service.cc:1467] Processing CreateTablet for tablet e46a760d7ab54f519a56e8d9cb046bef (DEFAULT_TABLE table=test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:12.098867 15193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46a760d7ab54f519a56e8d9cb046bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:12.100246 14902 tablet_service.cc:1467] Processing CreateTablet for tablet e46a760d7ab54f519a56e8d9cb046bef (DEFAULT_TABLE table=test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:12.101639 14902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e46a760d7ab54f519a56e8d9cb046bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:12.110850 15291 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Bootstrap starting.
I20260929 02:19:12.113452 15292 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Bootstrap starting.
I20260929 02:19:12.115509 15291 tablet_bootstrap.cc:654] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:12.116814 15291 log.cc:824] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:12.117806 15292 tablet_bootstrap.cc:654] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:12.119051 15292 log.cc:824] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:12.118240 15293 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Bootstrap starting.
I20260929 02:19:12.120134 15291 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: No bootstrap required, opened a new log
I20260929 02:19:12.120395 15291 ts_tablet_manager.cc:1397] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260929 02:19:12.121605 15292 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: No bootstrap required, opened a new log
I20260929 02:19:12.121842 15292 ts_tablet_manager.cc:1397] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260929 02:19:12.123422 15293 tablet_bootstrap.cc:654] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:12.124548 15293 log.cc:824] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:12.130016 15292 raft_consensus.cc:348] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.130403 15292 raft_consensus.cc:374] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:12.130492 15292 raft_consensus.cc:729] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e529507e6a5247abb705321d6a1a1700, State: Initialized, Role: FOLLOWER
I20260929 02:19:12.131080 15292 consensus_queue.cc:260] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.131335 15291 raft_consensus.cc:348] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.131762 15291 raft_consensus.cc:374] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:12.131855 15291 raft_consensus.cc:729] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4425660aad44c05acce0849cde31188, State: Initialized, Role: FOLLOWER
I20260929 02:19:12.132009 15293 tablet_bootstrap.cc:492] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: No bootstrap required, opened a new log
I20260929 02:19:12.132296 15293 ts_tablet_manager.cc:1397] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260929 02:19:12.132505 15291 consensus_queue.cc:260] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.137516 15291 ts_tablet_manager.cc:1428] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Time spent starting tablet: real 0.017s user 0.013s sys 0.004s
I20260929 02:19:12.137773 15260 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
I20260929 02:19:12.144423 15118 heartbeater.cc:503] Master 127.7.170.62:33267 was elected leader, sending a full tablet report...
I20260929 02:19:12.144701 15293 raft_consensus.cc:348] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.145117 15292 ts_tablet_manager.cc:1428] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Time spent starting tablet: real 0.023s user 0.007s sys 0.016s
I20260929 02:19:12.145218 15293 raft_consensus.cc:374] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:12.145313 15293 raft_consensus.cc:729] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de1c8f47e8a4d1bb7ee5420f09b711c, State: Initialized, Role: FOLLOWER
I20260929 02:19:12.145921 15293 consensus_queue.cc:260] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.152473 15293 ts_tablet_manager.cc:1428] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
W20260929 02:19:12.169450 15120 tablet.cc:2369] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:12.209960 15300 raft_consensus.cc:482] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:12.210228 15300 raft_consensus.cc:504] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.211779 15300 leader_election.cc:288] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351), e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:12.218820 15290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.212800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:45599 (local address 127.7.170.3:36599)
0929 02:19:12.213053 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.213059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.213402 (+ 343us) server_negotiation.cc:407] Connection header received
0929 02:19:12.213564 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.213569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.213642 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.213720 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:12.214285 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.214995 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.217148 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.217910 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.218009 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.218108 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.218467 (+ 359us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.218594 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:19:12.219024 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.212656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36599 (local address 127.7.170.1:45599)
0929 02:19:12.213304 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:12.213318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:12.213427 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:12.213748 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:12.213775 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:12.213802 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:12.214120 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.214132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.215122 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.215129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.217002 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.217015 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.218168 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.218179 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.218311 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.218351 (+ 40us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.218420 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.218546 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":6,"thread_start_us":383,"threads_started":1}
I20260929 02:19:12.219061 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.212020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37351 (local address 127.7.170.1:40183)
0929 02:19:12.212585 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:12.212604 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:12.212744 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:12.213145 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:12.213150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:12.213189 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:12.213555 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.213568 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.214945 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.214951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.216970 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:12.216992 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.218197 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.218203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:12.218324 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.218366 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 02:19:12.218449 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:19:12.218582 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":45,"thread_start_us":294,"threads_started":1}
I20260929 02:19:12.218851 15286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.212130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:40183 (local address 127.7.170.2:37351)
0929 02:19:12.212330 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.212337 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.212686 (+ 349us) server_negotiation.cc:407] Connection header received
0929 02:19:12.212879 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.212884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.212981 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.213066 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:12.213705 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.214644 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.217119 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.217958 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.218081 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.218169 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.218494 (+ 325us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.218612 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:19:12.220005 15072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46a760d7ab54f519a56e8d9cb046bef" candidate_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e529507e6a5247abb705321d6a1a1700" is_pre_election: true
I20260929 02:19:12.220245 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46a760d7ab54f519a56e8d9cb046bef" candidate_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4425660aad44c05acce0849cde31188" is_pre_election: true
I20260929 02:19:12.220479 15072 raft_consensus.cc:2454] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2de1c8f47e8a4d1bb7ee5420f09b711c in term 0.
I20260929 02:19:12.220737 15214 raft_consensus.cc:2454] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2de1c8f47e8a4d1bb7ee5420f09b711c in term 0.
I20260929 02:19:12.221213 14853 leader_election.cc:302] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de1c8f47e8a4d1bb7ee5420f09b711c, e529507e6a5247abb705321d6a1a1700; no voters:
I20260929 02:19:12.221577 15300 raft_consensus.cc:2781] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:12.221697 15300 raft_consensus.cc:482] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:12.221838 15300 raft_consensus.cc:3037] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:12.223971 15300 raft_consensus.cc:504] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.224484 15300 leader_election.cc:288] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [CANDIDATE]: Term 1 election: Requested vote from peers e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351), e4425660aad44c05acce0849cde31188 (127.7.170.3:36599)
I20260929 02:19:12.224911 15072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46a760d7ab54f519a56e8d9cb046bef" candidate_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e529507e6a5247abb705321d6a1a1700"
I20260929 02:19:12.225003 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e46a760d7ab54f519a56e8d9cb046bef" candidate_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4425660aad44c05acce0849cde31188"
I20260929 02:19:12.225114 15072 raft_consensus.cc:3037] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:12.225184 15214 raft_consensus.cc:3037] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:12.227308 15214 raft_consensus.cc:2454] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de1c8f47e8a4d1bb7ee5420f09b711c in term 1.
I20260929 02:19:12.227344 15072 raft_consensus.cc:2454] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2de1c8f47e8a4d1bb7ee5420f09b711c in term 1.
I20260929 02:19:12.227638 14853 leader_election.cc:302] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2de1c8f47e8a4d1bb7ee5420f09b711c, e4425660aad44c05acce0849cde31188; no voters:
I20260929 02:19:12.227874 15300 raft_consensus.cc:2781] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:12.228711 15300 raft_consensus.cc:686] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 LEADER]: Becoming Leader. State: Replica: 2de1c8f47e8a4d1bb7ee5420f09b711c, State: Running, Role: LEADER
I20260929 02:19:12.229282 15300 consensus_queue.cc:237] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } }
I20260929 02:19:12.233990 13997 catalog_manager.cc:5495] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1). New cstate: current_term: 1 leader_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } health_report { overall_health: HEALTHY } } }
W20260929 02:19:12.239820 15262 tablet.cc:2369] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:12.286540 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.281418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.7.170.1:43815)
0929 02:19:12.281640 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.281647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.281713 (+ 66us) server_negotiation.cc:407] Connection header received
0929 02:19:12.281937 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.281940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.282027 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.282156 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:12.282561 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.283225 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.284493 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.284883 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.285021 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.285450 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:12.285984 (+ 534us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:12.286114 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.286210 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.286286 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:19:12.295094 15072 raft_consensus.cc:1261] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Refusing update from remote peer 2de1c8f47e8a4d1bb7ee5420f09b711c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:12.295094 15214 raft_consensus.cc:1261] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Refusing update from remote peer 2de1c8f47e8a4d1bb7ee5420f09b711c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:12.296053 15303 consensus_queue.cc:1035] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:12.296375 15300 consensus_queue.cc:1035] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:12.331223 13996 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:12.331516 13996 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:12.339075 13996 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c5774655bf943fdb21b7eac2c55d9b5: Sending DeleteTablet for 3 replicas of tablet e46a760d7ab54f519a56e8d9cb046bef
I20260929 02:19:12.340160 15051 tablet_service.cc:1514] Processing DeleteTablet for tablet e46a760d7ab54f519a56e8d9cb046bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:12 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:49084
I20260929 02:19:12.340508 15193 tablet_service.cc:1514] Processing DeleteTablet for tablet e46a760d7ab54f519a56e8d9cb046bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:12 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:57646
I20260929 02:19:12.341404 15316 tablet_replica.cc:330] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: stopping tablet replica
I20260929 02:19:12.341814 15317 tablet_replica.cc:330] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: stopping tablet replica
I20260929 02:19:12.341737 14902 tablet_service.cc:1514] Processing DeleteTablet for tablet e46a760d7ab54f519a56e8d9cb046bef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:12 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:57628
I20260929 02:19:12.341964 15316 raft_consensus.cc:2229] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:12.342203 15317 raft_consensus.cc:2229] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:12.342548 15316 raft_consensus.cc:2258] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:12.344900 15316 ts_tablet_manager.cc:1905] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:12.349997 15317 raft_consensus.cc:2258] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:12.351953 15318 tablet_replica.cc:330] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: stopping tablet replica
I20260929 02:19:12.352415 15317 ts_tablet_manager.cc:1905] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:12.352385 15318 raft_consensus.cc:2229] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:19:12.353148 15318 raft_consensus.cc:2258] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:12.353153 15316 ts_tablet_manager.cc:1918] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:12.353394 15316 log.cc:1196] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/e46a760d7ab54f519a56e8d9cb046bef
I20260929 02:19:12.354007 15316 ts_tablet_manager.cc:1939] T e46a760d7ab54f519a56e8d9cb046bef P e4425660aad44c05acce0849cde31188: Deleting consensus metadata
I20260929 02:19:12.355296 15318 ts_tablet_manager.cc:1905] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:12.355949 13981 catalog_manager.cc:4841] TS e4425660aad44c05acce0849cde31188 (127.7.170.3:36599): tablet e46a760d7ab54f519a56e8d9cb046bef (table test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]) successfully deleted
I20260929 02:19:12.357823 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 13965
I20260929 02:19:12.358824 14027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:12.364462 15318 ts_tablet_manager.cc:1918] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:12.364635 15318 log.cc:1196] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/e46a760d7ab54f519a56e8d9cb046bef
I20260929 02:19:12.365238 15318 ts_tablet_manager.cc:1939] T e46a760d7ab54f519a56e8d9cb046bef P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting consensus metadata
I20260929 02:19:12.371613 15317 ts_tablet_manager.cc:1918] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:12.371802 15317 log.cc:1196] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/e46a760d7ab54f519a56e8d9cb046bef
I20260929 02:19:12.372350 15317 ts_tablet_manager.cc:1939] T e46a760d7ab54f519a56e8d9cb046bef P e529507e6a5247abb705321d6a1a1700: Deleting consensus metadata
I20260929 02:19:12.377969 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.343219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.7.170.61:41339)
0929 02:19:12.343382 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.343387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.355445 (+ 12058us) server_negotiation.cc:407] Connection header received
0929 02:19:12.357450 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.357458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.357577 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.357648 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.362229 (+ 4581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.363398 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.365442 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.366153 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.366255 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.369980 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.369987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.369992 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.370072 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.373610 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.374446 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.374452 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.374456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.374607 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.375091 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.375097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.375101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.375454 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.375907 (+ 453us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.376529 (+ 622us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.376756 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:19:12.393296 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.356405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32974 (local address 127.7.170.60:33613)
0929 02:19:12.356586 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.356591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.377106 (+ 20515us) server_negotiation.cc:407] Connection header received
0929 02:19:12.377355 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.377361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.377444 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.377512 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.379573 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.380641 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.382501 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.383171 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.383289 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.386575 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.386581 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.386587 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.386626 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.390422 (+ 3796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.391186 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.391192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.391196 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.391346 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.391720 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.391725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.391728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.392117 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.392406 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.392667 (+ 261us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.392887 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:19:12.489979 13981 catalog_manager.cc:4841] TS e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351): tablet e46a760d7ab54f519a56e8d9cb046bef (table test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]) successfully deleted
I20260929 02:19:12.490859 13983 catalog_manager.cc:4841] TS 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815): tablet e46a760d7ab54f519a56e8d9cb046bef (table test-table [id=7588d5bd4a3f4a9aa080ee4e03202ea4]) successfully deleted
I20260929 02:19:12.491421 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 13965
I20260929 02:19:12.528044 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.513352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.7.170.61:41339)
0929 02:19:12.513576 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.513583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.514528 (+ 945us) server_negotiation.cc:407] Connection header received
0929 02:19:12.514715 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.514720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.514788 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.514868 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.516973 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.518131 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.519628 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.520066 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.520174 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.523002 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.523007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.523010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.523040 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.525630 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.526287 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.526293 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.526297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.526429 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.526757 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.526762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.526766 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.527032 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.527246 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.527553 (+ 307us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.527744 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:19:12.530242 15268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:12.514614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32978 (local address 127.7.170.60:33613)
0929 02:19:12.514806 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:12.514812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:12.515217 (+ 405us) server_negotiation.cc:407] Connection header received
0929 02:19:12.515357 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:12.515362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:12.515439 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:12.515540 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:12.517660 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.518731 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.520661 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:12.521230 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:12.521352 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:12.524260 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:12.524266 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:12.524271 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:12.524308 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:12.527533 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.528278 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.528283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.528287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.528425 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:12.528806 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:12.528811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:12.528814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:12.529173 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:12.529409 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:12.529808 (+ 399us) server_negotiation.cc:299] Negotiation successful
0929 02:19:12.529969 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:19:13.322409 15328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.321193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:52729)
0929 02:19:13.322024 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.322202 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":631,"threads_started":1}
I20260929 02:19:13.322683 15327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.321094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:49175)
0929 02:19:13.321770 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.321912 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":469,"threads_started":1}
W20260929 02:19:13.322901 15260 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:33267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
W20260929 02:19:13.323014 15118 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:33267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.326835 15329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.325923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:46959)
0929 02:19:13.326338 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.326473 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":206,"threads_started":1}
W20260929 02:19:13.327149 14969 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:33267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.838315 15330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1c5774655bf943fdb21b7eac2c55d9b5)
I20260929 02:19:13.838600 15330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } }
I20260929 02:19:13.839001 15331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1c5774655bf943fdb21b7eac2c55d9b5)
I20260929 02:19:13.839175 15331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } }
I20260929 02:19:13.840106 15330 leader_election.cc:288] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267), 361a34c5b77b4663aa883c294a36524e (127.7.170.61:41339)
I20260929 02:19:13.840997 15331 leader_election.cc:288] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267), 05a33d21143e4ec3aa50847b04adf037 (127.7.170.60:33613)
I20260929 02:19:13.842058 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.840152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46244)
0929 02:19:13.841521 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.841737 (+ 216us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":1083,"threads_started":1}
I20260929 02:19:13.852494 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.844119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46236)
0929 02:19:13.851939 (+ 7820us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.852169 (+ 230us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7708,"thread_start_us":7549,"threads_started":1}
W20260929 02:19:13.853020 14581 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
W20260929 02:19:13.853024 14770 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.860101 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.840256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:41339 (local address 127.0.0.1:49926)
0929 02:19:13.842235 (+ 1979us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.842254 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:13.842393 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:13.852888 (+ 10495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:13.852893 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:13.852918 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:13.853208 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.853221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.855245 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.855251 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.857451 (+ 2200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.857468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.859292 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.859299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.859505 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.859547 (+ 42us) client_negotiation.cc:769] Sending connection context
0929 02:19:13.859615 (+ 68us) client_negotiation.cc:240] Negotiation successful
0929 02:19:13.859768 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1892,"thread_start_us":4467,"threads_started":1}
I20260929 02:19:13.860265 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.840152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:33613 (local address 127.0.0.1:32982)
0929 02:19:13.850964 (+ 10812us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.850992 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:13.851140 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:13.852971 (+ 1831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:13.852975 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:13.853014 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:13.853340 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.853354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.855355 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.855361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.857671 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.857688 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.859460 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.859467 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.859648 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.859687 (+ 39us) client_negotiation.cc:769] Sending connection context
0929 02:19:13.859754 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:19:13.859891 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10670,"thread_start_us":11072,"threads_started":1}
I20260929 02:19:13.860985 15337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.851489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.7.170.61:41339)
0929 02:19:13.852551 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:13.852556 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:13.852574 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:19:13.852675 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:13.852679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:13.852744 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:13.853346 (+ 602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:13.853611 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.855085 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.857686 (+ 2601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.859059 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.860301 (+ 1242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.860411 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:13.860486 (+ 75us) server_negotiation.cc:299] Negotiation successful
0929 02:19:13.860614 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":937,"thread_start_us":661,"threads_started":1}
I20260929 02:19:13.861579 15335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.841516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32982 (local address 127.7.170.60:33613)
0929 02:19:13.852613 (+ 11097us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:13.852619 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:13.852636 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:13.852731 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:13.852735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:13.852803 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:13.853503 (+ 700us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:13.853723 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.855196 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.857898 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.859252 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.860619 (+ 1367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.860737 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:13.860787 (+ 50us) server_negotiation.cc:299] Negotiation successful
0929 02:19:13.860915 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10991,"thread_start_us":11248,"threads_started":1}
I20260929 02:19:13.861591 14607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a33d21143e4ec3aa50847b04adf037" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "361a34c5b77b4663aa883c294a36524e" is_pre_election: true
I20260929 02:19:13.861974 14607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05a33d21143e4ec3aa50847b04adf037 in term 1.
I20260929 02:19:13.862542 14773 leader_election.cc:302] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05a33d21143e4ec3aa50847b04adf037, 361a34c5b77b4663aa883c294a36524e; no voters:
I20260929 02:19:13.862914 15330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260929 02:19:13.862972 15332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.862631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46270)
0929 02:19:13.862772 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.862852 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260929 02:19:13.863023 15330 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1c5774655bf943fdb21b7eac2c55d9b5)
I20260929 02:19:13.863114 15330 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 1 FOLLOWER]: Advancing to term 2
W20260929 02:19:13.863122 14581 leader_election.cc:334] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267): Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.863885 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.863517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46284)
0929 02:19:13.863656 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.863736 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:19:13.864089 14770 leader_election.cc:334] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267): Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.864024 14798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "361a34c5b77b4663aa883c294a36524e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "05a33d21143e4ec3aa50847b04adf037" is_pre_election: true
I20260929 02:19:13.865585 15330 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } }
I20260929 02:19:13.865950 14798 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 361a34c5b77b4663aa883c294a36524e in current term 2: Already voted for candidate 05a33d21143e4ec3aa50847b04adf037 in this term.
I20260929 02:19:13.866187 15330 leader_election.cc:288] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 election: Requested vote from peers 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267), 361a34c5b77b4663aa883c294a36524e (127.7.170.61:41339)
I20260929 02:19:13.866650 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.866288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46292)
0929 02:19:13.866428 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.866533 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:19:13.866596 14607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05a33d21143e4ec3aa50847b04adf037" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "361a34c5b77b4663aa883c294a36524e"
I20260929 02:19:13.866828 14607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:19:13.866849 14584 leader_election.cc:302] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 361a34c5b77b4663aa883c294a36524e; no voters: 05a33d21143e4ec3aa50847b04adf037, 1c5774655bf943fdb21b7eac2c55d9b5
I20260929 02:19:13.867462 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.867148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46308)
0929 02:19:13.867283 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.867359 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:19:13.867635 14770 leader_election.cc:334] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267): Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
I20260929 02:19:13.869005 14607 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05a33d21143e4ec3aa50847b04adf037 in term 2.
I20260929 02:19:13.869150 15331 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:19:13.869347 14773 leader_election.cc:302] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05a33d21143e4ec3aa50847b04adf037, 361a34c5b77b4663aa883c294a36524e; no voters: 1c5774655bf943fdb21b7eac2c55d9b5
I20260929 02:19:13.869597 15330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:19:13.870000 15330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [term 2 LEADER]: Becoming Leader. State: Replica: 05a33d21143e4ec3aa50847b04adf037, State: Running, Role: LEADER
I20260929 02:19:13.870343 15330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } }
I20260929 02:19:13.872054 15340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05a33d21143e4ec3aa50847b04adf037. Latest consensus state: current_term: 2 leader_uuid: "05a33d21143e4ec3aa50847b04adf037" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:13.872406 15340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:13.872997 15342 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:13.876231 15342 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:13.877034 15342 catalog_manager.cc:1259] Loaded cluster ID: d97c463e1c9f4a388e1ef94280449af8
I20260929 02:19:13.877099 15342 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:13.878159 15342 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:13.878891 15342 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Loaded TSK: 0
I20260929 02:19:13.879658 15342 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:19:13.917913 14787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:13.924638 14787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:13.928519 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.928169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46318)
0929 02:19:13.928307 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.928389 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:19:13.928699 14607 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [term 2 FOLLOWER]: Refusing update from remote peer 05a33d21143e4ec3aa50847b04adf037: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260929 02:19:13.929456 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.929092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46332)
0929 02:19:13.929257 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.929323 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 02:19:13.929486 15340 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [LEADER]: Connected to new peer: Peer: permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260929 02:19:13.929761 14770 consensus_peers.cc:487] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 -> Peer 1c5774655bf943fdb21b7eac2c55d9b5 (127.7.170.62:33267): Couldn't send request to peer 1c5774655bf943fdb21b7eac2c55d9b5. Status: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260929 02:19:13.932144 15331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05a33d21143e4ec3aa50847b04adf037. Latest consensus state: current_term: 2 leader_uuid: "05a33d21143e4ec3aa50847b04adf037" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:13.932380 15331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:13.933730 15340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "05a33d21143e4ec3aa50847b04adf037" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:13.934003 15340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:13.937070 15344 mvcc.cc:205] Tried to move back new op lower bound from 7334495657685762048 to 7334495657455857664. Current Snapshot: MvccSnapshot[applied={T|T < 7334495657685762048}]
I20260929 02:19:13.937704 15331 sys_catalog.cc:455] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "05a33d21143e4ec3aa50847b04adf037" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "1c5774655bf943fdb21b7eac2c55d9b5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 33267 } } peers { permanent_uuid: "361a34c5b77b4663aa883c294a36524e" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 41339 } attrs { promote: false } } peers { permanent_uuid: "05a33d21143e4ec3aa50847b04adf037" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 33613 } attrs { promote: false } } }
I20260929 02:19:13.937951 15331 sys_catalog.cc:458] T 00000000000000000000000000000000 P 361a34c5b77b4663aa883c294a36524e [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:19:13.959908 15333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.942396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36599 (local address 127.0.0.1:57656)
0929 02:19:13.942846 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.942864 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:13.942968 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:13.954451 (+ 11483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:13.954456 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:13.954484 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:13.954703 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.954714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.956006 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.956012 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.957587 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.957602 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.959174 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.959180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.959344 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.959377 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:19:13.959465 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:19:13.959592 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345}
I20260929 02:19:13.959930 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.942564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.7.170.3:36599)
0929 02:19:13.954132 (+ 11568us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:13.954139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:13.954159 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:19:13.954231 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:13.954237 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:13.954324 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:13.954426 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:13.954845 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.955885 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.957763 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.959026 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.959158 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.959271 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:13.959504 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 02:19:13.959632 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11445,"thread_start_us":11303,"threads_started":1}
I20260929 02:19:13.961053 15193 tablet_service.cc:1467] Processing CreateTablet for tablet 39651a8de7d5437495de7fb30c38d7ca (DEFAULT_TABLE table=test-table [id=633abc968f1c4f0fb650b514c33b681e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:13.961163 15334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.942846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:43815 (local address 127.0.0.1:57648)
0929 02:19:13.943110 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.943131 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:13.943230 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:13.955748 (+ 12518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:13.955753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:13.955777 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:13.956144 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.956159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.957620 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.957629 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.958885 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.958897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.960353 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.960359 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.960548 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.960580 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:19:13.960641 (+ 61us) client_negotiation.cc:240] Negotiation successful
0929 02:19:13.960807 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":25}
I20260929 02:19:13.961484 15193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39651a8de7d5437495de7fb30c38d7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:13.961666 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.942860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57648 (local address 127.7.170.1:43815)
0929 02:19:13.955462 (+ 12602us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:13.955467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:13.955504 (+ 37us) server_negotiation.cc:407] Connection header received
0929 02:19:13.955564 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:13.955569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:13.955631 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:13.955724 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:13.956369 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.957466 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.959027 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.960144 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.960829 (+ 685us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.960921 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:13.961250 (+ 329us) server_negotiation.cc:299] Negotiation successful
0929 02:19:13.961387 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12490,"thread_start_us":12945,"threads_started":1}
I20260929 02:19:13.962347 14902 tablet_service.cc:1467] Processing CreateTablet for tablet 39651a8de7d5437495de7fb30c38d7ca (DEFAULT_TABLE table=test-table [id=633abc968f1c4f0fb650b514c33b681e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:13.962731 14902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39651a8de7d5437495de7fb30c38d7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:13.965546 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.943077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:37351 (local address 127.0.0.1:49086)
0929 02:19:13.953316 (+ 10239us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.953335 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:13.953462 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:13.960624 (+ 7162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:13.960628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:13.960651 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:13.960999 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.961010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.962137 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.962142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.963349 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:13.963360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.964852 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.964858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:13.965041 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.965076 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:19:13.965156 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:19:13.965268 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10108,"mutex_wait_us":1,"thread_start_us":9929,"threads_started":1}
I20260929 02:19:13.966133 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.943516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49086 (local address 127.7.170.2:37351)
0929 02:19:13.960342 (+ 16826us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:13.960348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:13.960365 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:13.960429 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:13.960434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:13.960514 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:13.960600 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:13.961177 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.962036 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.963492 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:13.964668 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:13.964775 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:13.964860 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:13.965823 (+ 963us) server_negotiation.cc:299] Negotiation successful
0929 02:19:13.965924 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15897,"thread_start_us":16646,"threads_started":1}
I20260929 02:19:13.966733 15051 tablet_service.cc:1467] Processing CreateTablet for tablet 39651a8de7d5437495de7fb30c38d7ca (DEFAULT_TABLE table=test-table [id=633abc968f1c4f0fb650b514c33b681e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:13.967096 15051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39651a8de7d5437495de7fb30c38d7ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:13.967751 15353 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Bootstrap starting.
I20260929 02:19:13.969309 15354 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Bootstrap starting.
I20260929 02:19:13.969818 15353 tablet_bootstrap.cc:654] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:13.970546 15354 tablet_bootstrap.cc:654] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:13.971834 15353 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: No bootstrap required, opened a new log
I20260929 02:19:13.971963 15353 ts_tablet_manager.cc:1397] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260929 02:19:13.972774 15353 raft_consensus.cc:348] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.973042 15353 raft_consensus.cc:374] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:13.973136 15353 raft_consensus.cc:729] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4425660aad44c05acce0849cde31188, State: Initialized, Role: FOLLOWER
I20260929 02:19:13.973146 15357 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Bootstrap starting.
I20260929 02:19:13.973392 15353 consensus_queue.cc:260] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.974370 15354 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: No bootstrap required, opened a new log
I20260929 02:19:13.974546 15354 ts_tablet_manager.cc:1397] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260929 02:19:13.975064 15357 tablet_bootstrap.cc:654] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:13.975705 15354 raft_consensus.cc:348] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.976001 15354 raft_consensus.cc:374] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:13.976135 15354 raft_consensus.cc:729] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2de1c8f47e8a4d1bb7ee5420f09b711c, State: Initialized, Role: FOLLOWER
I20260929 02:19:13.976475 15353 ts_tablet_manager.cc:1428] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260929 02:19:13.976410 15354 consensus_queue.cc:260] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.977340 15357 tablet_bootstrap.cc:492] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: No bootstrap required, opened a new log
I20260929 02:19:13.977486 15357 ts_tablet_manager.cc:1397] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260929 02:19:13.978178 15357 raft_consensus.cc:348] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.978406 15357 raft_consensus.cc:374] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:13.978508 15357 raft_consensus.cc:729] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e529507e6a5247abb705321d6a1a1700, State: Initialized, Role: FOLLOWER
I20260929 02:19:13.978734 15357 consensus_queue.cc:260] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:13.979485 15357 ts_tablet_manager.cc:1428] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:13.979620 15259 heartbeater.cc:503] Master 127.7.170.60:33613 was elected leader, sending a full tablet report...
I20260929 02:19:13.983680 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.982648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:46123)
0929 02:19:13.983104 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.983412 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":577,"threads_started":1}
I20260929 02:19:13.984575 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.980773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:58455)
0929 02:19:13.984198 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.984325 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3293,"threads_started":1}
I20260929 02:19:13.984911 15117 heartbeater.cc:503] Master 127.7.170.60:33613 was elected leader, sending a full tablet report...
I20260929 02:19:13.985523 15354 ts_tablet_manager.cc:1428] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20260929 02:19:13.985706 14968 heartbeater.cc:503] Master 127.7.170.60:33613 was elected leader, sending a full tablet report...
I20260929 02:19:13.987680 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.987325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:56275)
0929 02:19:13.987460 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.987549 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:19:13.989581 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.977068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:46299)
0929 02:19:13.989259 (+ 12191us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.989375 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12084,"thread_start_us":11987,"threads_started":1}
I20260929 02:19:13.990378 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:13.990104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:58087)
0929 02:19:13.990224 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:13.990283 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:19:14.046144 15362 raft_consensus.cc:482] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:14.046393 15362 raft_consensus.cc:504] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:14.047926 15362 leader_election.cc:288] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4425660aad44c05acce0849cde31188 (127.7.170.3:36599), 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:14.055256 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.048312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:40221 (local address 127.7.170.1:43815)
0929 02:19:14.048507 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.048512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.048991 (+ 479us) server_negotiation.cc:407] Connection header received
0929 02:19:14.049243 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.049249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.049321 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.049397 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:14.049856 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.050773 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.053045 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.054226 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.054403 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.054502 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.054871 (+ 369us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.055005 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:14.055254 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.048312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42621 (local address 127.7.170.3:36599)
0929 02:19:14.048485 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.048491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.048517 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:19:14.048763 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.048769 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.048847 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.049002 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:14.049461 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.050541 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.053018 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.054238 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.054353 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.054452 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.054871 (+ 419us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.055005 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:19:14.055364 15365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.048248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:43815 (local address 127.7.170.2:40221)
0929 02:19:14.048883 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.048899 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:14.049092 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:14.049433 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:14.049437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:14.049460 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:14.049724 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:14.049732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.050883 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.050888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:14.052859 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:14.052868 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.054434 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.054439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:14.054672 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.054726 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 02:19:14.054812 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:19:14.054958 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":56,"thread_start_us":382,"threads_started":1}
I20260929 02:19:14.055680 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.048184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36599 (local address 127.7.170.2:42621)
0929 02:19:14.048436 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.048455 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:14.048606 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:14.048997 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:14.049001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:14.049022 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:14.049295 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:14.049307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.050655 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.050659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:14.052854 (+ 2195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:14.052868 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.054426 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.054433 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:14.054666 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.054720 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 02:19:14.054812 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 02:19:14.054956 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":69}
I20260929 02:19:14.056278 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39651a8de7d5437495de7fb30c38d7ca" candidate_uuid: "e529507e6a5247abb705321d6a1a1700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4425660aad44c05acce0849cde31188" is_pre_election: true
I20260929 02:19:14.056278 14922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39651a8de7d5437495de7fb30c38d7ca" candidate_uuid: "e529507e6a5247abb705321d6a1a1700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" is_pre_election: true
I20260929 02:19:14.056566 15214 raft_consensus.cc:2454] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e529507e6a5247abb705321d6a1a1700 in term 0.
I20260929 02:19:14.056633 14922 raft_consensus.cc:2454] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e529507e6a5247abb705321d6a1a1700 in term 0.
I20260929 02:19:14.057096 15002 leader_election.cc:302] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4425660aad44c05acce0849cde31188, e529507e6a5247abb705321d6a1a1700; no voters:
I20260929 02:19:14.057463 15362 raft_consensus.cc:2781] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:14.057581 15362 raft_consensus.cc:482] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:14.057667 15362 raft_consensus.cc:3037] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:14.059666 15362 raft_consensus.cc:504] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:14.060182 15362 leader_election.cc:288] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [CANDIDATE]: Term 1 election: Requested vote from peers e4425660aad44c05acce0849cde31188 (127.7.170.3:36599), 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815)
I20260929 02:19:14.060648 14922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39651a8de7d5437495de7fb30c38d7ca" candidate_uuid: "e529507e6a5247abb705321d6a1a1700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c"
I20260929 02:19:14.060648 15214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39651a8de7d5437495de7fb30c38d7ca" candidate_uuid: "e529507e6a5247abb705321d6a1a1700" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4425660aad44c05acce0849cde31188"
I20260929 02:19:14.060838 14922 raft_consensus.cc:3037] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:14.060838 15214 raft_consensus.cc:3037] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:14.063647 15214 raft_consensus.cc:2454] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e529507e6a5247abb705321d6a1a1700 in term 1.
I20260929 02:19:14.063647 14922 raft_consensus.cc:2454] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e529507e6a5247abb705321d6a1a1700 in term 1.
I20260929 02:19:14.064036 15006 leader_election.cc:302] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2de1c8f47e8a4d1bb7ee5420f09b711c, e4425660aad44c05acce0849cde31188, e529507e6a5247abb705321d6a1a1700; no voters:
I20260929 02:19:14.064321 15362 raft_consensus.cc:2781] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:14.064738 15362 raft_consensus.cc:686] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 1 LEADER]: Becoming Leader. State: Replica: e529507e6a5247abb705321d6a1a1700, State: Running, Role: LEADER
I20260929 02:19:14.065169 15362 consensus_queue.cc:237] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } }
I20260929 02:19:14.065623 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.065266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:57003)
0929 02:19:14.065377 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.065491 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260929 02:19:14.068521 14785 catalog_manager.cc:5495] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 reported cstate change: term changed from 0 to 1, leader changed from <none> to e529507e6a5247abb705321d6a1a1700 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "e529507e6a5247abb705321d6a1a1700" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e529507e6a5247abb705321d6a1a1700" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 37351 } health_report { overall_health: HEALTHY } } }
I20260929 02:19:14.084045 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.079505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49092 (local address 127.7.170.2:37351)
0929 02:19:14.079675 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.079680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.079696 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:14.079868 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.079872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.079947 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.080076 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:14.080337 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.081083 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.082324 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.082730 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.082822 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.083117 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:14.083619 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:14.083720 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.083757 (+ 37us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.083814 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260929 02:19:14.086097 14971 tablet.cc:2369] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:14.087898 15214 raft_consensus.cc:1261] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Refusing update from remote peer e529507e6a5247abb705321d6a1a1700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:14.088220 14922 raft_consensus.cc:1261] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Refusing update from remote peer e529507e6a5247abb705321d6a1a1700: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:14.088554 15366 consensus_queue.cc:1035] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4425660aad44c05acce0849cde31188" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:14.088876 15362 consensus_queue.cc:1035] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2de1c8f47e8a4d1bb7ee5420f09b711c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 43815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:14.091471 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.090478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:52815)
0929 02:19:14.091199 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.091317 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":592}
I20260929 02:19:14.091611 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.091079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:60753)
0929 02:19:14.091316 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.091418 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260929 02:19:14.094631 15369 mvcc.cc:205] Tried to move back new op lower bound from 7334495658337353728 to 7334495658254843904. Current Snapshot: MvccSnapshot[applied={T|T < 7334495658337353728}]
I20260929 02:19:14.095733 15370 mvcc.cc:205] Tried to move back new op lower bound from 7334495658337353728 to 7334495658254843904. Current Snapshot: MvccSnapshot[applied={T|T < 7334495658337353728}]
I20260929 02:19:14.096364 15361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.095833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.3:45143)
0929 02:19:14.096099 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.096182 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260929 02:19:14.101969 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.101439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:49877)
0929 02:19:14.101693 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.101789 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260929 02:19:14.110523 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.110011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.2:34969)
0929 02:19:14.110262 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.110344 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 02:19:14.111382 14787 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:14.111577 14787 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:14.115751 15364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.115339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.7.170.1:46097)
0929 02:19:14.115505 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.115596 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:19:14.116725 14787 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05a33d21143e4ec3aa50847b04adf037: Sending DeleteTablet for 3 replicas of tablet 39651a8de7d5437495de7fb30c38d7ca
I20260929 02:19:14.118114 15193 tablet_service.cc:1514] Processing DeleteTablet for tablet 39651a8de7d5437495de7fb30c38d7ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:14 UTC) from {username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:57656
I20260929 02:19:14.118104 14902 tablet_service.cc:1514] Processing DeleteTablet for tablet 39651a8de7d5437495de7fb30c38d7ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:14 UTC) from {username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:57648
I20260929 02:19:14.118764 15051 tablet_service.cc:1514] Processing DeleteTablet for tablet 39651a8de7d5437495de7fb30c38d7ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:14 UTC) from {username='oryx', principal='oryx/127.7.170.60@KRBTEST.COM'} at 127.0.0.1:49086
I20260929 02:19:14.121574 15371 mvcc.cc:205] Tried to move back new op lower bound from 7334495658337353728 to 7334495658254843904. Current Snapshot: MvccSnapshot[applied={T|T < 7334495658337353728}]
I20260929 02:19:14.122010 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14761
I20260929 02:19:14.138767 15380 tablet_replica.cc:330] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: stopping tablet replica
I20260929 02:19:14.141337 15380 raft_consensus.cc:2229] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:14.142083 15380 raft_consensus.cc:2258] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:14.142638 15379 tablet_replica.cc:330] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: stopping tablet replica
I20260929 02:19:14.142822 15379 raft_consensus.cc:2229] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:14.143121 15379 raft_consensus.cc:2258] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:14.144057 15380 ts_tablet_manager.cc:1905] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:14.145301 15381 tablet_replica.cc:330] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: stopping tablet replica
I20260929 02:19:14.145596 15381 raft_consensus.cc:2229] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:19:14.146034 15379 ts_tablet_manager.cc:1905] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:14.150435 15380 ts_tablet_manager.cc:1918] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:14.150884 15380 log.cc:1196] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/39651a8de7d5437495de7fb30c38d7ca
I20260929 02:19:14.151007 15379 ts_tablet_manager.cc:1918] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:14.151130 15379 log.cc:1196] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/39651a8de7d5437495de7fb30c38d7ca
I20260929 02:19:14.151343 15380 ts_tablet_manager.cc:1939] T 39651a8de7d5437495de7fb30c38d7ca P 2de1c8f47e8a4d1bb7ee5420f09b711c: Deleting consensus metadata
I20260929 02:19:14.151571 15379 ts_tablet_manager.cc:1939] T 39651a8de7d5437495de7fb30c38d7ca P e4425660aad44c05acce0849cde31188: Deleting consensus metadata
I20260929 02:19:14.152379 14772 catalog_manager.cc:4841] TS 2de1c8f47e8a4d1bb7ee5420f09b711c (127.7.170.1:43815): tablet 39651a8de7d5437495de7fb30c38d7ca (table test-table [id=633abc968f1c4f0fb650b514c33b681e]) successfully deleted
I20260929 02:19:14.152685 14770 catalog_manager.cc:4841] TS e4425660aad44c05acce0849cde31188 (127.7.170.3:36599): tablet 39651a8de7d5437495de7fb30c38d7ca (table test-table [id=633abc968f1c4f0fb650b514c33b681e]) successfully deleted
I20260929 02:19:14.153481 15381 raft_consensus.cc:2258] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:14.154310 15381 ts_tablet_manager.cc:1905] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:14.158941 15381 ts_tablet_manager.cc:1918] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:14.159060 15381 log.cc:1196] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/39651a8de7d5437495de7fb30c38d7ca
I20260929 02:19:14.159428 15381 ts_tablet_manager.cc:1939] T 39651a8de7d5437495de7fb30c38d7ca P e529507e6a5247abb705321d6a1a1700: Deleting consensus metadata
I20260929 02:19:14.160308 14770 catalog_manager.cc:4841] TS e529507e6a5247abb705321d6a1a1700 (127.7.170.2:37351): tablet 39651a8de7d5437495de7fb30c38d7ca (table test-table [id=633abc968f1c4f0fb650b514c33b681e]) successfully deleted
I20260929 02:19:14.162220 15350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.122927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.7.170.1:43815)
0929 02:19:14.123071 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.123078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.134122 (+ 11044us) server_negotiation.cc:407] Connection header received
0929 02:19:14.134306 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.134312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.134393 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.134492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:14.136101 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.137267 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.141286 (+ 4019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.141902 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.142019 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.155536 (+ 13517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:14.155553 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:14.155574 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:14.155612 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:14.159356 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.160204 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.160214 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.160217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.160373 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.160763 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.160768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.160770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.161156 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:14.161425 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.161725 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.161919 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260929 02:19:14.162828 14963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:14.302052 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14761
I20260929 02:19:14.328276 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14980
I20260929 02:19:14.347404 15352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.329003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49106 (local address 127.7.170.2:37351)
0929 02:19:14.329155 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.329159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.329241 (+ 82us) server_negotiation.cc:407] Connection header received
0929 02:19:14.329418 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.329422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.329484 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.329555 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:14.331208 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.332140 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.334009 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.334468 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.334565 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.341589 (+ 7024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:14.341602 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:14.341619 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:14.341654 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:14.344432 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.345308 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.345316 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.345318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.345502 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.345928 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.345931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.345933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.346382 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:14.346606 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.346932 (+ 326us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.347103 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:19:14.347997 15112 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:14.403702 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.403310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46346)
0929 02:19:14.403478 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.403563 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 02:19:14.404609 15351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.404283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:33267 (local address 127.0.0.1:46352)
0929 02:19:14.404417 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:14.404489 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:33267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:19:14.470935 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14980
I20260929 02:19:14.486542 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 15130
I20260929 02:19:14.498864 15349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:14.487217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57658 (local address 127.7.170.3:36599)
0929 02:19:14.487380 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:14.487385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:14.487438 (+ 53us) server_negotiation.cc:407] Connection header received
0929 02:19:14.487614 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:14.487620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:14.487686 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:14.487764 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:14.489066 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.489769 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.490911 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:14.491316 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:14.491423 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:14.493791 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:14.493805 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:14.493820 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:14.493862 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:14.496266 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.497008 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.497020 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.497023 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.497196 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:14.497591 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:14.497595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:14.497597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:14.497967 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:14.498180 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:14.498386 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:19:14.498546 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:14.499497 15254 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:14.589402 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 15130
I20260929 02:19:14.603320 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14460
I20260929 02:19:14.604331 14628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:14.717123 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14460
I20260929 02:19:14.734442 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 14563
I20260929 02:19:14.735491 14818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:14.851727 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 14563
2026-09-29T02:19:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9069 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:14 dist-test-slave-8cs2 krb5kdc[15390](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:14 dist-test-slave-8cs2 krb5kdc[15390](info): set up 2 sockets
Sep 29 02:19:14 dist-test-slave-8cs2 krb5kdc[15390](info): commencing operation
krb5kdc: starting...
W20260929 02:19:15.879359 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.988s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:16 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab.
Entry for principal oryx/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab.
I20260929 02:19:16.072293 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38163
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab
--principal=oryx/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:16.377444 15403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:16.377707 15403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:16.377808 15403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:16.384727 15403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:16.384859 15403 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260929 02:19:16.384939 15403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:16.385030 15403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:16.385097 15403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:16.385175 15403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:16.394840 15403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38163
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.62.keytab
--principal=oryx/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:16.396057 15403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:16.397356 15403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:16.408280 15408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:16.408371 15409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:16.408280 15411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:16.409392 15403 server_base.cc:1034] running on GCE node
I20260929 02:19:16.410046 15403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:16.411276 15403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:16.412488 15403 hybrid_clock.cc:648] HybridClock initialized: now 1790648356412425 us; error 37 us; skew 500 ppm
I20260929 02:19:16.412788 15403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:16 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:16.415575 15403 init.cc:377] Logged in from keytab as oryx/127.7.170.62@KRBTEST.COM (short username oryx)
I20260929 02:19:16.416882 15403 webserver.cc:460] Webserver started at http://127.7.170.62:37169/ using document root <none> and password file <none>
I20260929 02:19:16.417564 15403 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:16.417660 15403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:16.417954 15403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:16.420714 15403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "3c77415acec1439da908774dac0871e8"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "cf59f33c27ff36d84ff6b5d4af9942df"
server_key_iv: "607e8d263d47b73b680acc61ac8e17d0"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.421504 15403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "3c77415acec1439da908774dac0871e8"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "cf59f33c27ff36d84ff6b5d4af9942df"
server_key_iv: "607e8d263d47b73b680acc61ac8e17d0"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.426477 15403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:19:16.429625 15418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:16.430802 15403 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:19:16.430945 15403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "3c77415acec1439da908774dac0871e8"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "cf59f33c27ff36d84ff6b5d4af9942df"
server_key_iv: "607e8d263d47b73b680acc61ac8e17d0"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.431146 15403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:16.444743 15403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:16.448463 15403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:16.448709 15403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:16.467926 15403 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38163
I20260929 02:19:16.467945 15480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38163 every 8 connection(s)
I20260929 02:19:16.469308 15403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:16.472862 15481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:16.479651 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 15403
I20260929 02:19:16.479853 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:16.480612 7848 external_mini_cluster.cc:1428] Setting key e573d9160dd51cf265dc9ffe85b368f5
I20260929 02:19:16.483466 15481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Bootstrap starting.
I20260929 02:19:16.486828 15481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:16.487957 15481 log.cc:824] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:16.490955 15481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: No bootstrap required, opened a new log
Sep 29 02:19:16 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:16.499325 15481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } }
I20260929 02:19:16.499667 15481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:16.499783 15481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c77415acec1439da908774dac0871e8, State: Initialized, Role: FOLLOWER
I20260929 02:19:16.500452 15481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } }
I20260929 02:19:16.500730 15481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:16.500870 15481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:16.501091 15481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:16.502816 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:16.482597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39344 (local address 127.7.170.62:38163)
0929 02:19:16.483317 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:16.483329 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:16.483370 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:19:16.485115 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:16.485138 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:16.485507 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:16.485836 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:16.487455 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:16.488811 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:16.490483 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:16.491176 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:16.491306 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:16.495597 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:16.495606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:16.495625 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:16.495660 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:16.498578 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:16.499197 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:16.499204 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:16.499210 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:16.499373 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:16.499709 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:16.499714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:16.499717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:16.500389 (+ 672us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:16.500686 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:16.501050 (+ 364us) server_negotiation.cc:299] Negotiation successful
0929 02:19:16.501478 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":336,"threads_started":1}
I20260929 02:19:16.504179 15481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } }
I20260929 02:19:16.504662 15481 leader_election.cc:302] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c77415acec1439da908774dac0871e8; no voters:
I20260929 02:19:16.505218 15481 leader_election.cc:288] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:16.505352 15486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:16.506300 15486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [term 1 LEADER]: Becoming Leader. State: Replica: 3c77415acec1439da908774dac0871e8, State: Running, Role: LEADER
I20260929 02:19:16.506742 15486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } }
I20260929 02:19:16.507120 15481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:16.509070 15487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c77415acec1439da908774dac0871e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } } }
I20260929 02:19:16.509410 15487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:16.510043 15488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c77415acec1439da908774dac0871e8. Latest consensus state: current_term: 1 leader_uuid: "3c77415acec1439da908774dac0871e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c77415acec1439da908774dac0871e8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38163 } } }
I20260929 02:19:16.510285 15488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:16.511634 15492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:16.515908 15492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:16.523504 15492 catalog_manager.cc:1347] Generated new cluster ID: eda123b5916d4a0e911ccc0f33c060a0
I20260929 02:19:16.523602 15492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:16.539448 15492 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:16.540804 15492 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:16.548800 15492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Generated new TSK 0
I20260929 02:19:16.549831 15492 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
I20260929 02:19:16.601850 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38163
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:16.919235 15507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:16.919524 15507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:16.919658 15507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:16.926774 15507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:16.926875 15507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:16.926985 15507 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:19:16.927089 15507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:16.936653 15507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:16.937985 15507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:16.939262 15507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:16.951143 15512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:16.951212 15513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:16.951216 15515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:16.952039 15507 server_base.cc:1034] running on GCE node
I20260929 02:19:16.952548 15507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:16.953395 15507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:16.954594 15507 hybrid_clock.cc:648] HybridClock initialized: now 1790648356954537 us; error 31 us; skew 500 ppm
I20260929 02:19:16.954838 15507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:16 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:16.957645 15507 init.cc:377] Logged in from keytab as oryx/127.7.170.1@KRBTEST.COM (short username oryx)
I20260929 02:19:16.958957 15507 webserver.cc:460] Webserver started at http://127.7.170.1:38409/ using document root <none> and password file <none>
I20260929 02:19:16.959586 15507 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:16.959678 15507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:16.959959 15507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:16.962702 15507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "9a71bf91b65544e8aa0ac47076223291"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "0d2f1e1b168315b46a7be2e79caf03e7"
server_key_iv: "996a8b24f961fd354dbf28d7c39ebc42"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.963431 15507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "9a71bf91b65544e8aa0ac47076223291"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "0d2f1e1b168315b46a7be2e79caf03e7"
server_key_iv: "996a8b24f961fd354dbf28d7c39ebc42"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.968187 15507 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:19:16.971338 15522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:16.972523 15507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:16.972661 15507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "9a71bf91b65544e8aa0ac47076223291"
format_stamp: "Formatted at 2026-09-29 02:19:16 on dist-test-slave-8cs2"
server_key: "0d2f1e1b168315b46a7be2e79caf03e7"
server_key_iv: "996a8b24f961fd354dbf28d7c39ebc42"
server_key_version: "encryptionkey@0"
I20260929 02:19:16.972841 15507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:16.994915 15507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:16.998569 15507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:16.998796 15507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:17.001536 15507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:17.001617 15507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:17.001756 15507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:17.001814 15507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:19:17 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:17.023937 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.002197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:47649 (local address 127.7.170.62:38163)
0929 02:19:17.002382 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:17.002389 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:17.003845 (+ 1456us) server_negotiation.cc:407] Connection header received
0929 02:19:17.006469 (+ 2624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:17.006475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:17.006562 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:17.006678 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:17.009050 (+ 2372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.009988 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.011791 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.012451 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.012582 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.017493 (+ 4911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:17.017500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:17.017505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:17.017548 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:17.020573 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.021492 (+ 919us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.021498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.021502 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.021660 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.022227 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.022230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.022233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.022508 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:17.022614 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:17.023535 (+ 921us) server_negotiation.cc:299] Negotiation successful
0929 02:19:17.023707 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:19:17.025635 15531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.002778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.1:47649)
0929 02:19:17.003598 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:17.003650 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:17.006125 (+ 2475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:17.007185 (+ 1060us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:17.007200 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:17.007995 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:17.008785 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.008830 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.010138 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.010144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.011632 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.011644 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.012922 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.012950 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.013289 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.014149 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:17.014185 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:17.017236 (+ 3051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:17.020698 (+ 3462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.020707 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.020729 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.021369 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.021787 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.021794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.021798 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.022100 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.022616 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:17.022624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:17.023115 (+ 491us) client_negotiation.cc:769] Sending connection context
0929 02:19:17.023422 (+ 307us) client_negotiation.cc:240] Negotiation successful
0929 02:19:17.024006 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":322,"threads_started":1}
I20260929 02:19:17.044988 15600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.031681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.1:37699)
0929 02:19:17.032246 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:17.032260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:17.032378 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:17.032662 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:17.032667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:17.033122 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:17.033508 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.033528 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.034839 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.034844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.036546 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.036564 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.037606 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.037612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.037735 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.038353 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:17.038373 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:17.039142 (+ 769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:17.042356 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.042365 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.042369 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.042881 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.043210 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.043215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.043218 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.043376 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.044028 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:17.044031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:17.044200 (+ 169us) client_negotiation.cc:769] Sending connection context
0929 02:19:17.044361 (+ 161us) client_negotiation.cc:240] Negotiation successful
0929 02:19:17.044641 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":329,"threads_started":1}
I20260929 02:19:17.045765 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.031605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:37699 (local address 127.7.170.62:38163)
0929 02:19:17.031740 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:17.031744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:17.032340 (+ 596us) server_negotiation.cc:407] Connection header received
0929 02:19:17.032488 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:17.032494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:17.032556 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:17.032628 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:17.033709 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.034723 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.036682 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.037248 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.037365 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.039363 (+ 1998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:17.039369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:17.039375 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:17.039410 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:17.042255 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.042986 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.042991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.042995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.043115 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.043511 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.043517 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.043520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.043831 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:17.043963 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:17.045263 (+ 1300us) server_negotiation.cc:299] Negotiation successful
0929 02:19:17.045458 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:19:17.046322 15507 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:39021
I20260929 02:19:17.046343 15642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:39021 every 8 connection(s)
I20260929 02:19:17.047796 15507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:17.049829 15643 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38163
I20260929 02:19:17.050040 15643 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:17.050593 15643 heartbeater.cc:511] Master 127.7.170.62:38163 requested a full tablet report, sending...
I20260929 02:19:17.050800 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 15507
I20260929 02:19:17.050972 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:17.051728 7848 external_mini_cluster.cc:1428] Setting key 270534313ca93f9e4051c8cdb68529cd
I20260929 02:19:17.052619 15436 ts_manager.cc:194] Registered new tserver with Master: 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021)
I20260929 02:19:17.054366 15436 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:47649
WARNING: no policy specified for oryx/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
I20260929 02:19:17.093633 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38163
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:17.402438 15647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:17.402700 15647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:17.402832 15647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:17.409853 15647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:17.409973 15647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:17.410091 15647 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:19:17.410192 15647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:17.420245 15647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:17.421541 15647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:17.422787 15647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:17.435093 15652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:17.435153 15655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:17.435129 15653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:17.436105 15647 server_base.cc:1034] running on GCE node
I20260929 02:19:17.436662 15647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:17.437543 15647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:17.438776 15647 hybrid_clock.cc:648] HybridClock initialized: now 1790648357438719 us; error 27 us; skew 500 ppm
I20260929 02:19:17.439069 15647 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:17 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:17.441977 15647 init.cc:377] Logged in from keytab as oryx/127.7.170.2@KRBTEST.COM (short username oryx)
I20260929 02:19:17.443267 15647 webserver.cc:460] Webserver started at http://127.7.170.2:35475/ using document root <none> and password file <none>
I20260929 02:19:17.443933 15647 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:17.444027 15647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:17.444315 15647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:17.447103 15647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "91a00edef7674f4e9b6c061df3057f78"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "71187fbfd15b1d0e8dadc4ea8d3d1032"
server_key_iv: "d98e0d81c38c4931e51d2c2ee1dd5590"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.447840 15647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "91a00edef7674f4e9b6c061df3057f78"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "71187fbfd15b1d0e8dadc4ea8d3d1032"
server_key_iv: "d98e0d81c38c4931e51d2c2ee1dd5590"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.452704 15647 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:19:17.455936 15662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:17.457257 15647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:19:17.457401 15647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "91a00edef7674f4e9b6c061df3057f78"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "71187fbfd15b1d0e8dadc4ea8d3d1032"
server_key_iv: "d98e0d81c38c4931e51d2c2ee1dd5590"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.457599 15647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:17.469544 15647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:17.474349 15647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:17.474606 15647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:17.477491 15647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:17.477577 15647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:17.477700 15647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:17.477764 15647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:19:17 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:17.497784 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.477882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54165 (local address 127.7.170.62:38163)
0929 02:19:17.478060 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:17.478066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:17.479351 (+ 1285us) server_negotiation.cc:407] Connection header received
0929 02:19:17.481651 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:17.481656 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:17.481730 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:17.481812 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:17.483409 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.484462 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.487265 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.487709 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.487830 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.491563 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:17.491568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:17.491572 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:17.491602 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:17.494700 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.495441 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.495447 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.495451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.495592 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.496014 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.496018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.496022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.496299 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:17.496448 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:17.497387 (+ 939us) server_negotiation.cc:299] Negotiation successful
0929 02:19:17.497573 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:19:17.499469 15671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.478407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.2:54165)
0929 02:19:17.479203 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:17.479252 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:17.481384 (+ 2132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:17.482056 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:17.482064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:17.482563 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:17.483246 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.483264 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.484677 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.484682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.487030 (+ 2348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.487042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.487954 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.487963 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.488160 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.488738 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:17.488759 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:17.491353 (+ 2594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:17.494844 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.494851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.494864 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.495312 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.495698 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.495703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.495707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.495900 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.496452 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:17.496462 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:17.496969 (+ 507us) client_negotiation.cc:769] Sending connection context
0929 02:19:17.497297 (+ 328us) client_negotiation.cc:240] Negotiation successful
0929 02:19:17.497856 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":346,"threads_started":1}
I20260929 02:19:17.519595 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.505437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:45089 (local address 127.7.170.62:38163)
0929 02:19:17.505642 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:17.505648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:17.506334 (+ 686us) server_negotiation.cc:407] Connection header received
0929 02:19:17.506532 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:17.506538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:17.506615 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:17.506708 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:17.507725 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.508853 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.510125 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.510763 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.510890 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.513372 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:17.513380 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:17.513385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:17.513426 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:17.516705 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.517441 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.517448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.517451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.517599 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:17.517928 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:17.517934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:17.517938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:17.518357 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:17.518530 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:17.518982 (+ 452us) server_negotiation.cc:299] Negotiation successful
0929 02:19:17.519221 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:17.519773 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.505461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.2:45089)
0929 02:19:17.506218 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:17.506240 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:17.506392 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:17.506840 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:17.506843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:17.507284 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:17.507599 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.507609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.509014 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.509018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.510008 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.510019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.510955 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.510961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.511175 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.511925 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:17.511947 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:17.513167 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:17.516890 (+ 3723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.516896 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.516900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.517324 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.517698 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.517702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.517704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:17.517817 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:17.518485 (+ 668us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:17.518491 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:17.518705 (+ 214us) client_negotiation.cc:769] Sending connection context
0929 02:19:17.518879 (+ 174us) client_negotiation.cc:240] Negotiation successful
0929 02:19:17.519252 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":414,"threads_started":1}
I20260929 02:19:17.524693 15647 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:33105
I20260929 02:19:17.524714 15781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:33105 every 8 connection(s)
I20260929 02:19:17.525862 15647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:17.527818 15782 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38163
I20260929 02:19:17.528017 15782 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:17.528658 15782 heartbeater.cc:511] Master 127.7.170.62:38163 requested a full tablet report, sending...
I20260929 02:19:17.529902 15436 ts_manager.cc:194] Registered new tserver with Master: 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105)
I20260929 02:19:17.530665 15436 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:54165
I20260929 02:19:17.531113 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 15647
I20260929 02:19:17.531248 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:17.531652 7848 external_mini_cluster.cc:1428] Setting key 5b325595fb713724a787eec0a7173a18
WARNING: no policy specified for oryx/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
I20260929 02:19:17.576052 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38163
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:17.886965 15786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:17.887254 15786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:17.887393 15786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:17.894583 15786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:17.894696 15786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:17.894814 15786 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260929 02:19:17.894919 15786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:17.904601 15786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.170.62:38163
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:17.905889 15786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:17.907231 15786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:17.919729 15791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:17.919724 15792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:17.919956 15794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:17.920281 15786 server_base.cc:1034] running on GCE node
I20260929 02:19:17.920962 15786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:17.921960 15786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:17.923177 15786 hybrid_clock.cc:648] HybridClock initialized: now 1790648357923117 us; error 39 us; skew 500 ppm
I20260929 02:19:17.923460 15786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:17 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:17.926388 15786 init.cc:377] Logged in from keytab as oryx/127.7.170.3@KRBTEST.COM (short username oryx)
I20260929 02:19:17.927709 15786 webserver.cc:460] Webserver started at http://127.7.170.3:37801/ using document root <none> and password file <none>
I20260929 02:19:17.928498 15786 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:17.928592 15786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:17.928881 15786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:17.931695 15786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "c14e13a625a144cbb95b88568976c6b3"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "d261fd5287cfb2ec45c10f1e424b07bd"
server_key_iv: "309f7c9172b4c169866c91cf03e8aed0"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.932461 15786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "c14e13a625a144cbb95b88568976c6b3"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "d261fd5287cfb2ec45c10f1e424b07bd"
server_key_iv: "309f7c9172b4c169866c91cf03e8aed0"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.937407 15786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:19:17.940590 15801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:17.941879 15786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:17.942011 15786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "c14e13a625a144cbb95b88568976c6b3"
format_stamp: "Formatted at 2026-09-29 02:19:17 on dist-test-slave-8cs2"
server_key: "d261fd5287cfb2ec45c10f1e424b07bd"
server_key_iv: "309f7c9172b4c169866c91cf03e8aed0"
server_key_version: "encryptionkey@0"
I20260929 02:19:17.942178 15786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:17.973877 15786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:17.977775 15786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:17.977994 15786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:17.980469 15786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:17.980553 15786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:17.980695 15786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:17.980749 15786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:19:17 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648357, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:18.002720 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.981187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:47139 (local address 127.7.170.62:38163)
0929 02:19:17.982200 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:17.982207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:17.982756 (+ 549us) server_negotiation.cc:407] Connection header received
0929 02:19:17.985119 (+ 2363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:17.985124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:17.985208 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:17.985311 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:17.988050 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.988912 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.991127 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.991793 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.991933 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.996327 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:17.996332 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:17.996336 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:17.996367 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:17.999578 (+ 3211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.000289 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.000295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.000299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.000469 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.000889 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.000894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.000898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.001268 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:18.001421 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.002066 (+ 645us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.002328 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":871}
I20260929 02:19:18.003613 15810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:17.981749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.3:47139)
0929 02:19:17.982570 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:17.982619 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:17.984724 (+ 2105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:17.985629 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:17.985645 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:17.986692 (+ 1047us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:17.987759 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.987792 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.989130 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.989139 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.990978 (+ 1839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:17.990993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:17.992001 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:17.992006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:17.992184 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:17.992900 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:17.992944 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:17.996111 (+ 3167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:17.999725 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:17.999734 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:17.999750 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.000177 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.000578 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:18.000581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:18.000583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.000787 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.001383 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:18.001389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:18.001721 (+ 332us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.001970 (+ 249us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.002433 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":384,"threads_started":1}
I20260929 02:19:18.024533 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.009958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.7.170.3:57243)
0929 02:19:18.010604 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.010623 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.010778 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.011105 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.011113 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.011761 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:18.012209 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.012224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.013697 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.013703 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.015303 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.015322 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.016216 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.016227 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.016407 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.017187 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:18.017216 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:18.018500 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:18.021707 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:18.021712 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:18.021717 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.022263 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.022681 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:18.022690 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:18.022693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.022857 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.023421 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:18.023425 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:18.023664 (+ 239us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.023845 (+ 181us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.024165 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":332,"threads_started":1}
I20260929 02:19:18.024951 15786 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:44503
I20260929 02:19:18.024951 15920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:44503 every 8 connection(s)
I20260929 02:19:18.025542 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.009968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:57243 (local address 127.7.170.62:38163)
0929 02:19:18.010168 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.010174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.010712 (+ 538us) server_negotiation.cc:407] Connection header received
0929 02:19:18.010901 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.010906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.010972 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.011049 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:18.012373 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.013544 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.015575 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.016031 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.016123 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.018758 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:18.018766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:18.018771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:18.018812 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:18.021590 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.022387 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.022394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.022398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.022560 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.022957 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.022961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.022964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.023318 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:18.023477 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.025149 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.025308 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:19:18.026316 15786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:18.028438 15921 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38163
I20260929 02:19:18.028673 15921 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:18.029452 15921 heartbeater.cc:511] Master 127.7.170.62:38163 requested a full tablet report, sending...
I20260929 02:19:18.030648 15436 ts_manager.cc:194] Registered new tserver with Master: c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503)
I20260929 02:19:18.031428 15436 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:47139
I20260929 02:19:18.036113 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 15786
I20260929 02:19:18.036253 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:18.036667 7848 external_mini_cluster.cc:1428] Setting key f84bd778ade598c66feb253468612d97
I20260929 02:19:18.039201 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:19:18.056900 15643 heartbeater.cc:503] Master 127.7.170.62:38163 was elected leader, sending a full tablet report...
Sep 29 02:19:18 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 29 02:19:18 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:18 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:19:18.087440 15484 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:18.078124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.7.170.62:38163)
0929 02:19:18.078285 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.078289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.078545 (+ 256us) server_negotiation.cc:407] Connection header received
0929 02:19:18.078688 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.078692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.078767 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.078850 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:18.080059 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.080799 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.081998 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.082429 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.082516 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.087171 (+ 4655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39352: BlockingRecv error: recv got EOF from 127.0.0.1:39352 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:18 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:18.103154 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.090678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.7.170.62:38163)
0929 02:19:18.090833 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.090839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.091121 (+ 282us) server_negotiation.cc:407] Connection header received
0929 02:19:18.091272 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.091276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.091346 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.091426 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:18.092502 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.093226 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.094409 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.094833 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.094944 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.098366 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:18.098370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:18.098374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:18.098407 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:18.100724 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.101371 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.101376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.101379 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.101498 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.101829 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.101834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.101837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.102111 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:18.102371 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.102641 (+ 270us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.102847 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:19:18.107084 15436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:18.113933 15436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:18.145430 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.125070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.7.170.3:44503)
0929 02:19:18.136548 (+ 11478us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.136558 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.137450 (+ 892us) server_negotiation.cc:407] Connection header received
0929 02:19:18.137578 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.137587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.137886 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.138316 (+ 430us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.138898 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.140507 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.143296 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.144219 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.144331 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.144561 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.144867 (+ 306us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.145048 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11317,"thread_start_us":11152,"threads_started":1}
I20260929 02:19:18.145426 15939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.124773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:44503 (local address 127.0.0.1:45120)
0929 02:19:18.136989 (+ 12216us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.137041 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.137234 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.138116 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.138124 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.138155 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.138710 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.138720 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.140624 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.140630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.143158 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.143170 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.144392 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.144399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.144657 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.144694 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.144808 (+ 114us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.145001 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11907,"mutex_wait_us":75,"thread_start_us":11769,"threads_started":1}
I20260929 02:19:18.147228 15943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.124893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.0.0.1:42900)
0929 02:19:18.139065 (+ 14172us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.139083 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.139678 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.140271 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.140275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.140296 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.140715 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.140728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.142349 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.142352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.144286 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.144300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.146468 (+ 2168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.146473 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.146626 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.146655 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.146736 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.146857 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14063,"mutex_wait_us":3,"thread_start_us":14545,"threads_started":1}
I20260929 02:19:18.147383 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.125474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42900 (local address 127.7.170.1:39021)
0929 02:19:18.126320 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.126331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.139181 (+ 12850us) server_negotiation.cc:407] Connection header received
0929 02:19:18.139890 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.139897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.140118 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.140274 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.140998 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.142186 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.144466 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.146019 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.146107 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.146289 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.146964 (+ 675us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.147121 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":515,"threads_started":1}
I20260929 02:19:18.149587 15940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.126604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:33105 (local address 127.0.0.1:47480)
0929 02:19:18.139626 (+ 13022us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.139639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.139748 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.140503 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.140507 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.140529 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.141224 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.141235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.142954 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.142959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.145172 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.145186 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.148853 (+ 3667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.148860 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.149042 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.149073 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.149138 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.149251 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12936,"mutex_wait_us":35,"thread_start_us":12228,"threads_started":1}
I20260929 02:19:18.149611 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.126769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47480 (local address 127.7.170.2:33105)
0929 02:19:18.139379 (+ 12610us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.139388 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.139738 (+ 350us) server_negotiation.cc:407] Connection header received
0929 02:19:18.139926 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.139933 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.140149 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.140324 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.141419 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.142820 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.145344 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.148675 (+ 3331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.148783 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.149007 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.149206 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.149364 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12467,"thread_start_us":12351,"threads_started":1}
I20260929 02:19:18.149282 15851 tablet_service.cc:1467] Processing CreateTablet for tablet e5e46753145b4798bdf743035e6e7ca9 (DEFAULT_TABLE table=test-table [id=f74dade0ba1440caa1d4641716365b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.150758 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e46753145b4798bdf743035e6e7ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.151067 15573 tablet_service.cc:1467] Processing CreateTablet for tablet e5e46753145b4798bdf743035e6e7ca9 (DEFAULT_TABLE table=test-table [id=f74dade0ba1440caa1d4641716365b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.152374 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e46753145b4798bdf743035e6e7ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.155895 15712 tablet_service.cc:1467] Processing CreateTablet for tablet e5e46753145b4798bdf743035e6e7ca9 (DEFAULT_TABLE table=test-table [id=f74dade0ba1440caa1d4641716365b7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.157194 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5e46753145b4798bdf743035e6e7ca9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.164758 15945 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Bootstrap starting.
I20260929 02:19:18.168473 15945 tablet_bootstrap.cc:654] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.169189 15947 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Bootstrap starting.
I20260929 02:19:18.170305 15945 log.cc:824] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:18.171603 15946 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Bootstrap starting.
I20260929 02:19:18.172050 15947 tablet_bootstrap.cc:654] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.173223 15947 log.cc:824] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:18.173527 15945 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: No bootstrap required, opened a new log
I20260929 02:19:18.173817 15945 ts_tablet_manager.cc:1397] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260929 02:19:18.174659 15946 tablet_bootstrap.cc:654] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.175529 15946 log.cc:824] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:18.178062 15947 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: No bootstrap required, opened a new log
I20260929 02:19:18.178282 15947 ts_tablet_manager.cc:1397] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260929 02:19:18.178385 15946 tablet_bootstrap.cc:492] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: No bootstrap required, opened a new log
I20260929 02:19:18.178622 15946 ts_tablet_manager.cc:1397] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260929 02:19:18.183679 15945 raft_consensus.cc:348] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.184099 15945 raft_consensus.cc:374] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.184186 15945 raft_consensus.cc:729] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.184865 15945 consensus_queue.cc:260] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.186604 15947 raft_consensus.cc:348] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.186993 15947 raft_consensus.cc:374] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.187088 15947 raft_consensus.cc:729] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.187716 15947 consensus_queue.cc:260] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.187952 15921 heartbeater.cc:503] Master 127.7.170.62:38163 was elected leader, sending a full tablet report...
I20260929 02:19:18.188596 15945 ts_tablet_manager.cc:1428] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260929 02:19:18.188380 15946 raft_consensus.cc:348] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.188764 15946 raft_consensus.cc:374] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.188845 15946 raft_consensus.cc:729] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.189340 15946 consensus_queue.cc:260] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.190651 15946 ts_tablet_manager.cc:1428] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Time spent starting tablet: real 0.012s user 0.010s sys 0.003s
I20260929 02:19:18.192909 15782 heartbeater.cc:503] Master 127.7.170.62:38163 was elected leader, sending a full tablet report...
I20260929 02:19:18.194041 15947 ts_tablet_manager.cc:1428] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260929 02:19:18.277658 15783 tablet.cc:2369] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:19:18.277981 15922 tablet.cc:2369] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:19:18.300393 15644 tablet.cc:2369] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:18.408689 15952 raft_consensus.cc:482] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:18.409042 15952 raft_consensus.cc:504] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.410728 15952 leader_election.cc:288] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503), 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021)
I20260929 02:19:18.419004 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.411376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:55157 (local address 127.7.170.1:39021)
0929 02:19:18.411572 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.411578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.411835 (+ 257us) server_negotiation.cc:407] Connection header received
0929 02:19:18.412071 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.412075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.412159 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.412251 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.412968 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.414068 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.416718 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.417927 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.418064 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.418166 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.418604 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.418731 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:19:18.418984 15941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.411185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:53641 (local address 127.7.170.3:44503)
0929 02:19:18.411367 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.411374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.412138 (+ 764us) server_negotiation.cc:407] Connection header received
0929 02:19:18.412242 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.412247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.412315 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.412396 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.413066 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.414078 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.416718 (+ 2640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.417928 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.418062 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.418166 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.418604 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.418731 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:19:18.419204 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.411102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:44503 (local address 127.7.170.2:53641)
0929 02:19:18.412046 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.412061 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.412149 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.412424 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.412432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.412495 (+ 63us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.412906 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.412916 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.414197 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.414203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.416561 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.416576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.418161 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.418167 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.418390 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.418487 (+ 97us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.418579 (+ 92us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.418732 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":809,"mutex_wait_us":172,"thread_start_us":624,"threads_started":1}
I20260929 02:19:18.419224 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.411030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.7.170.2:55157)
0929 02:19:18.411703 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.411724 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.411915 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.412284 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.412287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.412306 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.412743 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.412755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.414197 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.414203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.416570 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.416578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.418164 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.418169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.418394 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.418489 (+ 95us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.418579 (+ 90us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.418732 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":93,"spinlock_wait_cycles":28544,"thread_start_us":359,"threads_started":1}
I20260929 02:19:18.420199 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e46753145b4798bdf743035e6e7ca9" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291" is_pre_election: true
I20260929 02:19:18.420317 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e46753145b4798bdf743035e6e7ca9" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3" is_pre_election: true
I20260929 02:19:18.420671 15593 raft_consensus.cc:2454] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 0.
I20260929 02:19:18.420766 15875 raft_consensus.cc:2454] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 0.
I20260929 02:19:18.421295 15666 leader_election.cc:302] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291; no voters:
I20260929 02:19:18.421657 15952 raft_consensus.cc:2781] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:18.421788 15952 raft_consensus.cc:482] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:18.421883 15952 raft_consensus.cc:3037] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.423938 15952 raft_consensus.cc:504] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.424515 15952 leader_election.cc:288] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 election: Requested vote from peers c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503), 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021)
I20260929 02:19:18.424993 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e46753145b4798bdf743035e6e7ca9" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3"
I20260929 02:19:18.424993 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5e46753145b4798bdf743035e6e7ca9" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291"
I20260929 02:19:18.425194 15593 raft_consensus.cc:3037] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.425197 15875 raft_consensus.cc:3037] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.428283 15875 raft_consensus.cc:2454] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 1.
I20260929 02:19:18.428283 15593 raft_consensus.cc:2454] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 1.
I20260929 02:19:18.428733 15664 leader_election.cc:302] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:18.429122 15952 raft_consensus.cc:2781] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:18.429518 15952 raft_consensus.cc:686] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 1 LEADER]: Becoming Leader. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Running, Role: LEADER
I20260929 02:19:18.429951 15952 consensus_queue.cc:237] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } }
I20260929 02:19:18.434597 15433 catalog_manager.cc:5495] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "91a00edef7674f4e9b6c061df3057f78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } health_report { overall_health: HEALTHY } } }
I20260929 02:19:18.497437 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.492631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47496 (local address 127.7.170.2:33105)
0929 02:19:18.492843 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.492849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.492869 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:19:18.492995 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.492999 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.493076 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.493205 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:18.493538 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.494204 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.495435 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.495844 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.495949 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.496357 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:18.496877 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:18.497032 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.497128 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.497203 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:19:18.505228 15875 raft_consensus.cc:1261] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Refusing update from remote peer 91a00edef7674f4e9b6c061df3057f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:18.505235 15593 raft_consensus.cc:1261] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Refusing update from remote peer 91a00edef7674f4e9b6c061df3057f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:18.506131 15956 consensus_queue.cc:1035] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:18.506443 15952 consensus_queue.cc:1035] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:18.516563 15959 mvcc.cc:205] Tried to move back new op lower bound from 7334495676427968512 to 7334495676132704256. Current Snapshot: MvccSnapshot[applied={T|T < 7334495676427968512}]
I20260929 02:19:18.530550 15436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:18.530829 15436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:18.534487 15436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Sending DeleteTablet for 3 replicas of tablet e5e46753145b4798bdf743035e6e7ca9
I20260929 02:19:18.535492 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e46753145b4798bdf743035e6e7ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:18 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45120
I20260929 02:19:18.535779 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e46753145b4798bdf743035e6e7ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:18 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:42900
I20260929 02:19:18.536763 15712 tablet_service.cc:1514] Processing DeleteTablet for tablet e5e46753145b4798bdf743035e6e7ca9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:18 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47480
I20260929 02:19:18.541430 15970 tablet_replica.cc:330] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: stopping tablet replica
I20260929 02:19:18.541419 15436 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260929 02:19:18.541862 15970 raft_consensus.cc:2229] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 1 LEADER]: Raft consensus shutting down.
W20260929 02:19:18.542083 15436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:18.542604 15970 raft_consensus.cc:2258] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:18.545588 15968 tablet_replica.cc:330] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: stopping tablet replica
I20260929 02:19:18.546015 15968 raft_consensus.cc:2229] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:18.546582 15968 raft_consensus.cc:2258] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:18.549818 15572 tablet_service.cc:1467] Processing CreateTablet for tablet 12997194b72a43259f271d28219f16af (DEFAULT_TABLE table=test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.550179 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 12997194b72a43259f271d28219f16af (DEFAULT_TABLE table=test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.550367 15572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12997194b72a43259f271d28219f16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.550664 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12997194b72a43259f271d28219f16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.553375 15968 ts_tablet_manager.cc:1905] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:18.553611 15712 tablet_service.cc:1467] Processing CreateTablet for tablet 12997194b72a43259f271d28219f16af (DEFAULT_TABLE table=test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:18.554052 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12997194b72a43259f271d28219f16af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.557083 15945 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Bootstrap starting.
I20260929 02:19:18.558462 15945 tablet_bootstrap.cc:654] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.560060 15945 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: No bootstrap required, opened a new log
I20260929 02:19:18.560181 15945 ts_tablet_manager.cc:1397] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260929 02:19:18.560829 15945 raft_consensus.cc:348] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.561049 15945 raft_consensus.cc:374] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.561141 15945 raft_consensus.cc:729] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.561403 15945 consensus_queue.cc:260] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.561597 15946 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Bootstrap starting.
I20260929 02:19:18.563143 15946 tablet_bootstrap.cc:654] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.564167 15945 ts_tablet_manager.cc:1428] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:19:18.566706 15947 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Bootstrap starting.
I20260929 02:19:18.568348 15947 tablet_bootstrap.cc:654] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.569028 15968 ts_tablet_manager.cc:1918] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:18.569193 15968 log.cc:1196] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/e5e46753145b4798bdf743035e6e7ca9
I20260929 02:19:18.569653 15968 ts_tablet_manager.cc:1939] T e5e46753145b4798bdf743035e6e7ca9 P c14e13a625a144cbb95b88568976c6b3: Deleting consensus metadata
I20260929 02:19:18.570591 15970 ts_tablet_manager.cc:1905] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:18.571388 15420 catalog_manager.cc:4841] TS c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503): tablet e5e46753145b4798bdf743035e6e7ca9 (table test-table [id=f74dade0ba1440caa1d4641716365b7b]) successfully deleted
I20260929 02:19:18.576833 15970 ts_tablet_manager.cc:1918] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:18.580557 15969 tablet_replica.cc:330] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: stopping tablet replica
I20260929 02:19:18.581027 15969 raft_consensus.cc:2229] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:18.581533 15969 raft_consensus.cc:2258] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:18.582465 15947 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: No bootstrap required, opened a new log
I20260929 02:19:18.582652 15947 ts_tablet_manager.cc:1397] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.001s
I20260929 02:19:18.583242 15946 tablet_bootstrap.cc:492] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: No bootstrap required, opened a new log
I20260929 02:19:18.583369 15946 ts_tablet_manager.cc:1397] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.008s
I20260929 02:19:18.583494 15947 raft_consensus.cc:348] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.583722 15947 raft_consensus.cc:374] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.583818 15947 raft_consensus.cc:729] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.584136 15946 raft_consensus.cc:348] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.584113 15947 consensus_queue.cc:260] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.584331 15946 raft_consensus.cc:374] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.584422 15946 raft_consensus.cc:729] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.584717 15946 consensus_queue.cc:260] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.585330 15969 ts_tablet_manager.cc:1905] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:18.586124 15946 ts_tablet_manager.cc:1428] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:18.586756 15947 ts_tablet_manager.cc:1428] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260929 02:19:18.588029 15970 log.cc:1196] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/e5e46753145b4798bdf743035e6e7ca9
I20260929 02:19:18.588634 15970 ts_tablet_manager.cc:1939] T e5e46753145b4798bdf743035e6e7ca9 P 91a00edef7674f4e9b6c061df3057f78: Deleting consensus metadata
I20260929 02:19:18.590312 15419 catalog_manager.cc:4841] TS 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105): tablet e5e46753145b4798bdf743035e6e7ca9 (table test-table [id=f74dade0ba1440caa1d4641716365b7b]) successfully deleted
I20260929 02:19:18.593310 15969 ts_tablet_manager.cc:1918] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:18.593459 15969 log.cc:1196] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/e5e46753145b4798bdf743035e6e7ca9
I20260929 02:19:18.593938 15969 ts_tablet_manager.cc:1939] T e5e46753145b4798bdf743035e6e7ca9 P 9a71bf91b65544e8aa0ac47076223291: Deleting consensus metadata
I20260929 02:19:18.595503 15422 catalog_manager.cc:4841] TS 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021): tablet e5e46753145b4798bdf743035e6e7ca9 (table test-table [id=f74dade0ba1440caa1d4641716365b7b]) successfully deleted
I20260929 02:19:18.877343 15951 raft_consensus.cc:482] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:18.877563 15951 raft_consensus.cc:504] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.879096 15951 leader_election.cc:288] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105)
I20260929 02:19:18.885746 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.879508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:36037 (local address 127.7.170.1:39021)
0929 02:19:18.879700 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.879706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.880225 (+ 519us) server_negotiation.cc:407] Connection header received
0929 02:19:18.880550 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.880554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.880627 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.880705 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.881235 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.882208 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.884036 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.884881 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.885026 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.885130 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.885363 (+ 233us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.885489 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:19:18.885751 15975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.879292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.7.170.3:36037)
0929 02:19:18.880115 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.880137 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.880349 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.880769 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.880774 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.880799 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.881068 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.881081 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.882370 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.882375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.883889 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.883900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.885049 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.885055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.885222 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.885245 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.885324 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.885445 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"thread_start_us":976,"threads_started":1}
I20260929 02:19:18.886483 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12997194b72a43259f271d28219f16af" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291" is_pre_election: true
I20260929 02:19:18.886729 15593 raft_consensus.cc:2454] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 0.
I20260929 02:19:18.887382 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.879392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:33105 (local address 127.7.170.3:36839)
0929 02:19:18.880567 (+ 1175us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.880584 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.880719 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.881148 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.881153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.881179 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:18.881494 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.881505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.882882 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.882887 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.884979 (+ 2092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.885002 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.886667 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.886673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.886841 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.886871 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.886958 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.887076 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"thread_start_us":575,"threads_started":1}
I20260929 02:19:18.887403 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.879724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:36839 (local address 127.7.170.2:33105)
0929 02:19:18.879887 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.879894 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.880678 (+ 784us) server_negotiation.cc:407] Connection header received
0929 02:19:18.880900 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.880908 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.881026 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.881118 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:18.881670 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.882742 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.885176 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.886454 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.886608 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.886704 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.886998 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.887126 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:19:18.887290 15805 leader_election.cc:302] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a71bf91b65544e8aa0ac47076223291, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:18.887708 15951 raft_consensus.cc:2781] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:18.887812 15951 raft_consensus.cc:482] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:18.887902 15951 raft_consensus.cc:3037] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.888415 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12997194b72a43259f271d28219f16af" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78" is_pre_election: true
I20260929 02:19:18.888849 15737 raft_consensus.cc:2454] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 0.
I20260929 02:19:18.890738 15951 raft_consensus.cc:504] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.891283 15951 leader_election.cc:288] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 election: Requested vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105)
I20260929 02:19:18.891752 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12997194b72a43259f271d28219f16af" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78"
I20260929 02:19:18.891746 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12997194b72a43259f271d28219f16af" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291"
I20260929 02:19:18.891939 15593 raft_consensus.cc:3037] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.891947 15737 raft_consensus.cc:3037] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:18.894785 15593 raft_consensus.cc:2454] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 1.
I20260929 02:19:18.894886 15737 raft_consensus.cc:2454] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 1.
I20260929 02:19:18.895215 15805 leader_election.cc:302] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a71bf91b65544e8aa0ac47076223291, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:18.895509 15951 raft_consensus.cc:2781] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:18.895926 15951 raft_consensus.cc:686] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 1 LEADER]: Becoming Leader. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Running, Role: LEADER
I20260929 02:19:18.896306 15951 consensus_queue.cc:237] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.899550 15436 catalog_manager.cc:5495] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c14e13a625a144cbb95b88568976c6b3 (127.7.170.3). New cstate: current_term: 1 leader_uuid: "c14e13a625a144cbb95b88568976c6b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } health_report { overall_health: HEALTHY } } }
Sep 29 02:19:18 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648356, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.62@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:18.963071 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.950988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38163 (local address 127.0.0.1:39374)
0929 02:19:18.951622 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:18.951640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:18.951748 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:18.952045 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:18.952048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:18.952635 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:18.952807 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.952815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.953872 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.953876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.954927 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:18.954938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.955573 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.955577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:18.955692 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.956247 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:18.956263 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:18.958354 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:18.960897 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:18.960906 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:18.960912 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.961367 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.961698 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:18.961702 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:18.961706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:18.961843 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:18.962289 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:18.962295 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:18.962428 (+ 133us) client_negotiation.cc:769] Sending connection context
0929 02:19:18.962571 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 02:19:18.962760 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":323,"threads_started":1}
I20260929 02:19:18.963177 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:18.951011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39374 (local address 127.7.170.62:38163)
0929 02:19:18.951644 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:18.951648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:18.951688 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:19:18.951844 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:18.951848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:18.951902 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:18.951983 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:18.952956 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.953741 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.955047 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:18.955447 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:18.955531 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:18.958509 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:18.958514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:18.958517 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:18.958543 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:18.960796 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.961465 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.961469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.961471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.961590 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:18.961926 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:18.961929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:18.961931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:18.962180 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:18.962291 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:18.962658 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 02:19:18.962875 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":528,"thread_start_us":355,"threads_started":1}
I20260929 02:19:18.967564 15436 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:39374:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260929 02:19:18.968472 15436 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:18.974280 15573 tablet_service.cc:1467] Processing CreateTablet for tablet 19c0516173144bfbbfcd8b205f67ad90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61d4f6379d2e49bfb8db6e534829838b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:19:18.974476 15712 tablet_service.cc:1467] Processing CreateTablet for tablet 19c0516173144bfbbfcd8b205f67ad90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61d4f6379d2e49bfb8db6e534829838b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:19:18.974857 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c0516173144bfbbfcd8b205f67ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.974681 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 19c0516173144bfbbfcd8b205f67ad90 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=61d4f6379d2e49bfb8db6e534829838b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260929 02:19:18.974938 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c0516173144bfbbfcd8b205f67ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.975199 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19c0516173144bfbbfcd8b205f67ad90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:18.980520 15945 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: Bootstrap starting.
I20260929 02:19:18.981518 15947 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: Bootstrap starting.
I20260929 02:19:18.981524 15946 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: Bootstrap starting.
I20260929 02:19:18.982064 15945 tablet_bootstrap.cc:654] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.982993 15947 tablet_bootstrap.cc:654] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.983040 15946 tablet_bootstrap.cc:654] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:18.984149 15945 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: No bootstrap required, opened a new log
I20260929 02:19:18.984292 15945 ts_tablet_manager.cc:1397] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:19:18.984812 15947 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: No bootstrap required, opened a new log
I20260929 02:19:18.985019 15947 ts_tablet_manager.cc:1397] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:19:18.985389 15946 tablet_bootstrap.cc:492] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: No bootstrap required, opened a new log
I20260929 02:19:18.985261 15945 raft_consensus.cc:348] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.985496 15945 raft_consensus.cc:374] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.985544 15946 ts_tablet_manager.cc:1397] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260929 02:19:18.985569 15945 raft_consensus.cc:729] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.985674 15947 raft_consensus.cc:348] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.985899 15947 raft_consensus.cc:374] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.985842 15945 consensus_queue.cc:260] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.985962 15947 raft_consensus.cc:729] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.986233 15946 raft_consensus.cc:348] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.986187 15947 consensus_queue.cc:260] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.986222 15951 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:18.986413 15946 raft_consensus.cc:374] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:18.986454 15951 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:18.986486 15946 raft_consensus.cc:729] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Initialized, Role: FOLLOWER
I20260929 02:19:18.986562 15956 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:18.986809 15956 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:18.986699 15946 consensus_queue.cc:260] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:18.987124 15953 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:18.987463 15953 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:18.987612 15947 ts_tablet_manager.cc:1428] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:19:18.988070 15946 ts_tablet_manager.cc:1428] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:18.988526 15945 ts_tablet_manager.cc:1428] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:19:19.082316 15953 raft_consensus.cc:482] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:19.082531 15953 raft_consensus.cc:504] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.083897 15953 leader_election.cc:288] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105), c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503)
I20260929 02:19:19.089635 15944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.084190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:54993 (local address 127.7.170.2:33105)
0929 02:19:19.084368 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.084375 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.084827 (+ 452us) server_negotiation.cc:407] Connection header received
0929 02:19:19.085120 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.085125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.085184 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.085252 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:19.085705 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.086378 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.088040 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.088746 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.088884 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.089013 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.089240 (+ 227us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.089368 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:19:19.089751 15988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.084170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:33105 (local address 127.7.170.1:54993)
0929 02:19:19.084719 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.084746 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.084966 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.085281 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.085285 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.085321 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:19.085581 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.085590 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.086503 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.086506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.087898 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.087909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.088922 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.088948 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.089104 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.089128 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.089208 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.089331 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":1,"spinlock_wait_cycles":4352,"thread_start_us":367,"threads_started":1}
I20260929 02:19:19.090436 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.084325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:44257 (local address 127.7.170.3:44503)
0929 02:19:19.084997 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.085003 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.085287 (+ 284us) server_negotiation.cc:407] Connection header received
0929 02:19:19.085449 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.085453 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.085511 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.085589 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:19.086126 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.086836 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.088466 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.089320 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.089435 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.089533 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.090058 (+ 525us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.090188 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":434,"threads_started":1}
I20260929 02:19:19.090348 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c0516173144bfbbfcd8b205f67ad90" candidate_uuid: "9a71bf91b65544e8aa0ac47076223291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78" is_pre_election: true
I20260929 02:19:19.090502 15989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.084186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:44503 (local address 127.7.170.1:44257)
0929 02:19:19.085186 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.085200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.085323 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.085616 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.085620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.085642 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:19.085969 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.085979 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.086957 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.086962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.088324 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.088335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.089716 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.089725 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.089903 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.089933 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.090014 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.090130 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"spinlock_wait_cycles":3840,"thread_start_us":777,"threads_started":1}
I20260929 02:19:19.090611 15737 raft_consensus.cc:2454] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a71bf91b65544e8aa0ac47076223291 in term 0.
I20260929 02:19:19.091045 15523 leader_election.cc:302] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291; no voters:
I20260929 02:19:19.091109 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c0516173144bfbbfcd8b205f67ad90" candidate_uuid: "9a71bf91b65544e8aa0ac47076223291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3" is_pre_election: true
I20260929 02:19:19.091290 15875 raft_consensus.cc:2454] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a71bf91b65544e8aa0ac47076223291 in term 0.
I20260929 02:19:19.091372 15953 raft_consensus.cc:2781] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:19.091478 15953 raft_consensus.cc:482] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:19.091579 15953 raft_consensus.cc:3037] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.094038 15953 raft_consensus.cc:504] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.094612 15953 leader_election.cc:288] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [CANDIDATE]: Term 1 election: Requested vote from peers 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105), c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503)
I20260929 02:19:19.094928 15737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c0516173144bfbbfcd8b205f67ad90" candidate_uuid: "9a71bf91b65544e8aa0ac47076223291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78"
I20260929 02:19:19.095033 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19c0516173144bfbbfcd8b205f67ad90" candidate_uuid: "9a71bf91b65544e8aa0ac47076223291" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3"
I20260929 02:19:19.095115 15737 raft_consensus.cc:3037] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.095220 15875 raft_consensus.cc:3037] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.097059 15737 raft_consensus.cc:2454] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a71bf91b65544e8aa0ac47076223291 in term 1.
I20260929 02:19:19.097360 15875 raft_consensus.cc:2454] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a71bf91b65544e8aa0ac47076223291 in term 1.
I20260929 02:19:19.097419 15523 leader_election.cc:302] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291; no voters:
I20260929 02:19:19.097671 15953 raft_consensus.cc:2781] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:19.098019 15953 raft_consensus.cc:686] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [term 1 LEADER]: Becoming Leader. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Running, Role: LEADER
I20260929 02:19:19.098363 15953 consensus_queue.cc:237] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.100061 15991 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: TxnStatusTablet state changed. Reason: New leader 9a71bf91b65544e8aa0ac47076223291. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.100405 15991 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:19:19.101023 15993 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260929 02:19:19.101219 15993 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260929 02:19:19.101594 15433 catalog_manager.cc:5495] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1). New cstate: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } health_report { overall_health: UNKNOWN } } }
I20260929 02:19:19.140398 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.134509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.7.170.1:39021)
0929 02:19:19.134679 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.134684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.134705 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:19:19.135033 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.135037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.135130 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.135244 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.135636 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.136504 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.138368 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.138758 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.138844 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.139344 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.139896 (+ 552us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.139991 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.140112 (+ 121us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.140173 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:19:19.140556 15983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.134375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.0.0.1:42908)
0929 02:19:19.134643 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.134665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.134841 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.135291 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.135297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.135311 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:19:19.135510 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.135521 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.136624 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.136630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.138239 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.138255 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.138887 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.138894 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.139115 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.139186 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.139993 (+ 807us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.140013 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.140098 (+ 85us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.140243 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260929 02:19:19.144495 15875 raft_consensus.cc:1261] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Refusing update from remote peer 9a71bf91b65544e8aa0ac47076223291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.145004 15737 raft_consensus.cc:1261] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Refusing update from remote peer 9a71bf91b65544e8aa0ac47076223291: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.145164 15991 consensus_queue.cc:1035] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.145767 15953 consensus_queue.cc:1035] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.154881 15951 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: TxnStatusTablet state changed. Reason: New leader 9a71bf91b65544e8aa0ac47076223291. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.155166 15951 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:19.157223 15956 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: TxnStatusTablet state changed. Reason: New leader 9a71bf91b65544e8aa0ac47076223291. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.157517 15956 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:19.158461 15997 mvcc.cc:205] Tried to move back new op lower bound from 7334495679049596928 to 7334495678870327296. Current Snapshot: MvccSnapshot[applied={T|T < 7334495679049596928}]
I20260929 02:19:19.160952 15953 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.161244 15953 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:19:19.161509 15953 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.161718 15953 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 9a71bf91b65544e8aa0ac47076223291: This TxnStatusTablet replica's current role is: LEADER
I20260929 02:19:19.164700 15996 mvcc.cc:205] Tried to move back new op lower bound from 7334495679049596928 to 7334495678870327296. Current Snapshot: MvccSnapshot[applied={T|T < 7334495679049596928 or (T in {7334495679049596928})}]
I20260929 02:19:19.167478 15951 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.167783 15951 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P c14e13a625a144cbb95b88568976c6b3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:19.173691 15956 tablet_replica.cc:439] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9a71bf91b65544e8aa0ac47076223291" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } } }
I20260929 02:19:19.173974 15956 tablet_replica.cc:442] T 19c0516173144bfbbfcd8b205f67ad90 P 91a00edef7674f4e9b6c061df3057f78: This TxnStatusTablet replica's current role is: FOLLOWER
I20260929 02:19:19.181030 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.174187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45128 (local address 127.7.170.3:44503)
0929 02:19:19.174356 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.174362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.174631 (+ 269us) server_negotiation.cc:407] Connection header received
0929 02:19:19.174924 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.174930 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.175035 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.175156 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.175770 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.176899 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.178923 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.179329 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.179432 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.179923 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.180461 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.180566 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.180682 (+ 116us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.180747 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:19:19.198172 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.191578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:37213 (local address 127.7.170.1:39021)
0929 02:19:19.191718 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.191722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.193399 (+ 1677us) server_negotiation.cc:407] Connection header received
0929 02:19:19.193574 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.193577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.193641 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.193712 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.194210 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.194987 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.196437 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.196896 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.197035 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.197426 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.197542 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.197619 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.197758 (+ 139us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.197834 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260929 02:19:19.198421 16008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.191498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.7.170.3:37213)
0929 02:19:19.193299 (+ 1801us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.193314 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.193450 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.193736 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.193740 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.193751 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:19:19.194067 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.194077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.195121 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.195126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.196297 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.196310 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.197083 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.197090 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.197210 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.197262 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.197634 (+ 372us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.197653 (+ 19us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.197718 (+ 65us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.197861 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1672,"thread_start_us":1568,"threads_started":1}
I20260929 02:19:19.208706 15593 raft_consensus.cc:1261] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Refusing update from remote peer c14e13a625a144cbb95b88568976c6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.208732 15737 raft_consensus.cc:1261] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Refusing update from remote peer c14e13a625a144cbb95b88568976c6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.209391 15951 consensus_queue.cc:1035] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.209885 15951 consensus_queue.cc:1035] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.245261 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.236442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:46953 (local address 127.7.170.3:44503)
0929 02:19:19.236611 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.236617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.237293 (+ 676us) server_negotiation.cc:407] Connection header received
0929 02:19:19.238501 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.238507 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.238599 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.238662 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.239807 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.241068 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.243383 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.243883 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.243986 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.244465 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.244580 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.244654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.244837 (+ 183us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.244918 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:19:19.245396 16014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.236446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:44503 (local address 127.7.170.1:46953)
0929 02:19:19.237188 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.237211 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.237420 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.239130 (+ 1710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.239135 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.239150 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:19:19.239591 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.239602 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.241330 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.241336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.243261 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.243272 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.244050 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.244057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.244202 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.244268 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.244710 (+ 442us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.244721 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.244807 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.244997 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":325,"threads_started":1}
I20260929 02:19:19.289858 15433 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.290136 15433 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.293503 15433 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Sending DeleteTablet for 3 replicas of tablet 12997194b72a43259f271d28219f16af
I20260929 02:19:19.294494 15712 tablet_service.cc:1514] Processing DeleteTablet for tablet 12997194b72a43259f271d28219f16af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47480
I20260929 02:19:19.294533 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet 12997194b72a43259f271d28219f16af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45120
I20260929 02:19:19.295051 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet 12997194b72a43259f271d28219f16af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:42900
I20260929 02:19:19.317647 16019 tablet_replica.cc:330] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: stopping tablet replica
I20260929 02:19:19.318029 16019 raft_consensus.cc:2229] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.318581 16017 tablet_replica.cc:330] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: stopping tablet replica
I20260929 02:19:19.318956 16019 raft_consensus.cc:2258] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.319221 16017 raft_consensus.cc:2229] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.319640 16017 raft_consensus.cc:2258] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.321079 16018 tablet_replica.cc:330] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: stopping tablet replica
I20260929 02:19:19.321269 16018 raft_consensus.cc:2229] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:19:19.321864 16018 raft_consensus.cc:2258] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.320852 16019 ts_tablet_manager.cc:1905] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.322904 16018 ts_tablet_manager.cc:1905] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.323177 16017 ts_tablet_manager.cc:1905] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.330195 16018 ts_tablet_manager.cc:1918] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.330327 16018 log.cc:1196] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/12997194b72a43259f271d28219f16af
I20260929 02:19:19.330802 16018 ts_tablet_manager.cc:1939] T 12997194b72a43259f271d28219f16af P c14e13a625a144cbb95b88568976c6b3: Deleting consensus metadata
I20260929 02:19:19.331962 15420 catalog_manager.cc:4841] TS c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503): tablet 12997194b72a43259f271d28219f16af (table test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]) successfully deleted
I20260929 02:19:19.334064 16017 ts_tablet_manager.cc:1918] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.334249 16017 log.cc:1196] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/12997194b72a43259f271d28219f16af
I20260929 02:19:19.334648 16017 ts_tablet_manager.cc:1939] T 12997194b72a43259f271d28219f16af P 91a00edef7674f4e9b6c061df3057f78: Deleting consensus metadata
I20260929 02:19:19.335234 16019 ts_tablet_manager.cc:1918] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.335489 16019 log.cc:1196] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/12997194b72a43259f271d28219f16af
I20260929 02:19:19.335769 15419 catalog_manager.cc:4841] TS 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105): tablet 12997194b72a43259f271d28219f16af (table test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]) successfully deleted
I20260929 02:19:19.335948 16019 ts_tablet_manager.cc:1939] T 12997194b72a43259f271d28219f16af P 9a71bf91b65544e8aa0ac47076223291: Deleting consensus metadata
I20260929 02:19:19.336982 15422 catalog_manager.cc:4841] TS 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021): tablet 12997194b72a43259f271d28219f16af (table test-table [id=e4637d5e9fe24a5fbc752acd3c35b893]) successfully deleted
Sep 29 02:19:19 dist-test-slave-8cs2 krb5kdc[15390](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 29 02:19:19 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:19 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:19:19.393118 15984 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:19.374712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.7.170.62:38163)
0929 02:19:19.374910 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.374916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.375411 (+ 495us) server_negotiation.cc:407] Connection header received
0929 02:19:19.375591 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.375597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.375692 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.375798 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:19.377432 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.378514 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.380306 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.380995 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.381120 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.392818 (+ 11698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39386: BlockingRecv error: recv got EOF from 127.0.0.1:39386 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:19:19 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM
I20260929 02:19:19.414487 15984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.396334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.7.170.62:38163)
0929 02:19:19.396507 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.396512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.396997 (+ 485us) server_negotiation.cc:407] Connection header received
0929 02:19:19.397179 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.397185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.397263 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.397370 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:19.398862 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.400109 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.401844 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.402539 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.402662 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.407360 (+ 4698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:19.407367 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:19.407372 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:19.407414 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:19.411340 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:19.412172 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:19.412178 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:19.412181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:19.412366 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:19.412917 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:19.412922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:19.412946 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:19.413295 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:19.413543 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.413888 (+ 345us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.414123 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:19:19.418393 15433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:19.419085 15433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:19.426106 15573 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae37738933b409db3a8bf774799b6f4 (DEFAULT_TABLE table=test-table [id=5c5297c6ae71438689eb81f65ef9e426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.426575 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae37738933b409db3a8bf774799b6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.426488 15851 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae37738933b409db3a8bf774799b6f4 (DEFAULT_TABLE table=test-table [id=5c5297c6ae71438689eb81f65ef9e426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.426939 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae37738933b409db3a8bf774799b6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.429879 15712 tablet_service.cc:1467] Processing CreateTablet for tablet 9ae37738933b409db3a8bf774799b6f4 (DEFAULT_TABLE table=test-table [id=5c5297c6ae71438689eb81f65ef9e426]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.430281 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ae37738933b409db3a8bf774799b6f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.432468 15946 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Bootstrap starting.
I20260929 02:19:19.433902 15946 tablet_bootstrap.cc:654] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.435750 15946 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: No bootstrap required, opened a new log
I20260929 02:19:19.435883 15946 ts_tablet_manager.cc:1397] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260929 02:19:19.436579 15947 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Bootstrap starting.
I20260929 02:19:19.436546 15946 raft_consensus.cc:348] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.436791 15946 raft_consensus.cc:374] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.436880 15946 raft_consensus.cc:729] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.437125 15946 consensus_queue.cc:260] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.437645 15946 ts_tablet_manager.cc:1428] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:19.438047 15947 tablet_bootstrap.cc:654] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.445632 15945 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Bootstrap starting.
I20260929 02:19:19.447152 15947 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: No bootstrap required, opened a new log
I20260929 02:19:19.447289 15947 ts_tablet_manager.cc:1397] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260929 02:19:19.447373 15945 tablet_bootstrap.cc:654] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.448021 15947 raft_consensus.cc:348] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.448361 15947 raft_consensus.cc:374] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.448459 15947 raft_consensus.cc:729] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.448660 15947 consensus_queue.cc:260] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.452450 15947 ts_tablet_manager.cc:1428] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260929 02:19:19.456761 15945 tablet_bootstrap.cc:492] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: No bootstrap required, opened a new log
I20260929 02:19:19.456908 15945 ts_tablet_manager.cc:1397] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260929 02:19:19.457566 15945 raft_consensus.cc:348] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.457764 15945 raft_consensus.cc:374] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.457858 15945 raft_consensus.cc:729] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.458125 15945 consensus_queue.cc:260] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.458622 15945 ts_tablet_manager.cc:1428] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260929 02:19:19.500219 16012 raft_consensus.cc:482] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:19.500424 16012 raft_consensus.cc:504] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.501188 16012 leader_election.cc:288] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105)
I20260929 02:19:19.501767 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae37738933b409db3a8bf774799b6f4" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291" is_pre_election: true
I20260929 02:19:19.501749 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae37738933b409db3a8bf774799b6f4" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78" is_pre_election: true
I20260929 02:19:19.501992 15736 raft_consensus.cc:2454] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 0.
I20260929 02:19:19.501991 15593 raft_consensus.cc:2454] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 0.
I20260929 02:19:19.502542 15802 leader_election.cc:302] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:19.502839 16012 raft_consensus.cc:2781] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:19.502954 16012 raft_consensus.cc:482] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:19.503051 16012 raft_consensus.cc:3037] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.505740 16012 raft_consensus.cc:504] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.506785 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae37738933b409db3a8bf774799b6f4" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91a00edef7674f4e9b6c061df3057f78"
I20260929 02:19:19.506808 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ae37738933b409db3a8bf774799b6f4" candidate_uuid: "c14e13a625a144cbb95b88568976c6b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291"
I20260929 02:19:19.506997 15593 raft_consensus.cc:3037] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.506995 15736 raft_consensus.cc:3037] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.509812 15736 raft_consensus.cc:2454] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 1.
I20260929 02:19:19.509931 15593 raft_consensus.cc:2454] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c14e13a625a144cbb95b88568976c6b3 in term 1.
I20260929 02:19:19.510244 15802 leader_election.cc:302] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:19.510561 15976 raft_consensus.cc:2781] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:19.510720 15976 raft_consensus.cc:686] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 1 LEADER]: Becoming Leader. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Running, Role: LEADER
I20260929 02:19:19.511013 15976 consensus_queue.cc:237] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.513567 15433 catalog_manager.cc:5495] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c14e13a625a144cbb95b88568976c6b3 (127.7.170.3). New cstate: current_term: 1 leader_uuid: "c14e13a625a144cbb95b88568976c6b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } health_report { overall_health: HEALTHY } } }
I20260929 02:19:19.514423 16012 leader_election.cc:288] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [CANDIDATE]: Term 1 election: Requested vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105)
I20260929 02:19:19.532764 15990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.525639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45134 (local address 127.7.170.3:44503)
0929 02:19:19.525789 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.525813 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.525830 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:19.525896 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.525900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.525977 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.526057 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.526599 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.527699 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.529591 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.530240 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.530348 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.531173 (+ 825us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.531321 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.531419 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.532385 (+ 966us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.532465 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:19:19.535356 15593 raft_consensus.cc:1261] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Refusing update from remote peer c14e13a625a144cbb95b88568976c6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.535970 15976 consensus_queue.cc:1035] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.536568 15736 raft_consensus.cc:1261] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Refusing update from remote peer c14e13a625a144cbb95b88568976c6b3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.537143 15976 consensus_queue.cc:1035] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.562702 15436 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.562973 15436 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.566445 15436 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Sending DeleteTablet for 3 replicas of tablet 9ae37738933b409db3a8bf774799b6f4
I20260929 02:19:19.567118 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae37738933b409db3a8bf774799b6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:42900
I20260929 02:19:19.567404 16019 tablet_replica.cc:330] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: stopping tablet replica
I20260929 02:19:19.568003 15712 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae37738933b409db3a8bf774799b6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47480
I20260929 02:19:19.568260 16017 tablet_replica.cc:330] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: stopping tablet replica
I20260929 02:19:19.568444 16017 raft_consensus.cc:2229] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.568677 16017 raft_consensus.cc:2258] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.569391 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ae37738933b409db3a8bf774799b6f4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45120
I20260929 02:19:19.569655 16018 tablet_replica.cc:330] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: stopping tablet replica
I20260929 02:19:19.569845 16018 raft_consensus.cc:2229] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:19:19.569773 15435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:19.570546 15435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:19.572436 16017 ts_tablet_manager.cc:1905] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.578498 16017 ts_tablet_manager.cc:1918] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:19.578651 16017 log.cc:1196] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/9ae37738933b409db3a8bf774799b6f4
I20260929 02:19:19.579053 16017 ts_tablet_manager.cc:1939] T 9ae37738933b409db3a8bf774799b6f4 P 91a00edef7674f4e9b6c061df3057f78: Deleting consensus metadata
I20260929 02:19:19.579545 15573 tablet_service.cc:1467] Processing CreateTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 (DEFAULT_TABLE table=test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.579941 15573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8a4b243ff4446db4c3fd48b7489285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.580955 15419 catalog_manager.cc:4841] TS 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105): tablet 9ae37738933b409db3a8bf774799b6f4 (table test-table [id=5c5297c6ae71438689eb81f65ef9e426]) successfully deleted
I20260929 02:19:19.581420 15712 tablet_service.cc:1467] Processing CreateTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 (DEFAULT_TABLE table=test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.581777 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8a4b243ff4446db4c3fd48b7489285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.583012 16019 raft_consensus.cc:2229] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.583455 16019 pending_rounds.cc:62] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Trying to abort 2 pending ops.
I20260929 02:19:19.583570 16019 pending_rounds.cc:68] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334495680651128832 op_type: WRITE_OP write_request { tablet_id: "9ae37738933b409db3a8bf774799b6f4" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\323\302\354\325\006\"8\n\ntest-admin\022*\n 5c5297c6ae71438689eb81f65ef9e426\020\001\030\001 \001(\001" signature: "<redacted>""\263\337\300\256!K\003K\260\215\353\340\375\264\013\251\037\355ZzV\241\341\213\221\317\374\2222\001>ze\365\224\364\311\3678\365\231#\350Q\177\363\014\355LV\031\324\347\326;<-\216\364\r\336\"\243\030" signing_key_seq_num: 0 } } request_id { client_id: "9d53ddd2b121410bb5d22b968f4e0e78" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260929 02:19:19.584257 16019 pending_rounds.cc:68] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334495680559501312 op_type: NO_OP noop_request { }
I20260929 02:19:19.584407 16019 raft_consensus.cc:2866] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260929 02:19:19.584528 16019 raft_consensus.cc:2258] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.584833 16018 raft_consensus.cc:2258] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.585664 15851 tablet_service.cc:1467] Processing CreateTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 (DEFAULT_TABLE table=test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:19.586120 15851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca8a4b243ff4446db4c3fd48b7489285. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:19.589818 16018 ts_tablet_manager.cc:1905] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.595770 16018 ts_tablet_manager.cc:1918] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:19.595934 16018 log.cc:1196] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/9ae37738933b409db3a8bf774799b6f4
I20260929 02:19:19.596267 16018 ts_tablet_manager.cc:1939] T 9ae37738933b409db3a8bf774799b6f4 P c14e13a625a144cbb95b88568976c6b3: Deleting consensus metadata
I20260929 02:19:19.597117 15946 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Bootstrap starting.
I20260929 02:19:19.597187 15420 catalog_manager.cc:4841] TS c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503): tablet 9ae37738933b409db3a8bf774799b6f4 (table test-table [id=5c5297c6ae71438689eb81f65ef9e426]) successfully deleted
I20260929 02:19:19.598584 15946 tablet_bootstrap.cc:654] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.603364 15947 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Bootstrap starting.
I20260929 02:19:19.604137 15946 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: No bootstrap required, opened a new log
I20260929 02:19:19.604290 15946 ts_tablet_manager.cc:1397] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260929 02:19:19.604545 15945 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Bootstrap starting.
I20260929 02:19:19.604954 15947 tablet_bootstrap.cc:654] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.605235 16019 ts_tablet_manager.cc:1905] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.605883 15946 raft_consensus.cc:348] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.606215 15946 raft_consensus.cc:374] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.606329 15946 raft_consensus.cc:729] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a71bf91b65544e8aa0ac47076223291, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.606529 15945 tablet_bootstrap.cc:654] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:19.606680 15946 consensus_queue.cc:260] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.607247 15946 ts_tablet_manager.cc:1428] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:19.612038 16019 ts_tablet_manager.cc:1918] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:19.612231 16019 log.cc:1196] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/9ae37738933b409db3a8bf774799b6f4
I20260929 02:19:19.612593 16019 ts_tablet_manager.cc:1939] T 9ae37738933b409db3a8bf774799b6f4 P 9a71bf91b65544e8aa0ac47076223291: Deleting consensus metadata
I20260929 02:19:19.613754 15422 catalog_manager.cc:4841] TS 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021): tablet 9ae37738933b409db3a8bf774799b6f4 (table test-table [id=5c5297c6ae71438689eb81f65ef9e426]) successfully deleted
I20260929 02:19:19.617475 15947 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: No bootstrap required, opened a new log
I20260929 02:19:19.617645 15947 ts_tablet_manager.cc:1397] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.009s
I20260929 02:19:19.618239 15947 raft_consensus.cc:348] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.618427 15947 raft_consensus.cc:374] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.618495 15947 raft_consensus.cc:729] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.618698 15947 consensus_queue.cc:260] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.619359 15945 tablet_bootstrap.cc:492] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: No bootstrap required, opened a new log
I20260929 02:19:19.619519 15945 ts_tablet_manager.cc:1397] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.000s
I20260929 02:19:19.620115 15945 raft_consensus.cc:348] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.620306 15945 raft_consensus.cc:374] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:19.620391 15945 raft_consensus.cc:729] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c14e13a625a144cbb95b88568976c6b3, State: Initialized, Role: FOLLOWER
I20260929 02:19:19.620625 15945 consensus_queue.cc:260] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.621114 15945 ts_tablet_manager.cc:1428] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:19.621606 15947 ts_tablet_manager.cc:1428] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260929 02:19:19.866851 15956 raft_consensus.cc:482] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:19.867100 15956 raft_consensus.cc:504] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.867712 15956 leader_election.cc:288] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503)
I20260929 02:19:19.868257 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8a4b243ff4446db4c3fd48b7489285" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291" is_pre_election: true
I20260929 02:19:19.868279 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8a4b243ff4446db4c3fd48b7489285" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3" is_pre_election: true
I20260929 02:19:19.868464 15593 raft_consensus.cc:2454] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 0.
I20260929 02:19:19.868489 15875 raft_consensus.cc:2454] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 0.
I20260929 02:19:19.868848 15666 leader_election.cc:302] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291; no voters:
I20260929 02:19:19.869177 15956 raft_consensus.cc:2781] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:19.869275 15956 raft_consensus.cc:482] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:19.869385 15956 raft_consensus.cc:3037] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.871443 15956 raft_consensus.cc:504] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.871981 15956 leader_election.cc:288] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 election: Requested vote from peers 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021), c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503)
I20260929 02:19:19.872418 15593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8a4b243ff4446db4c3fd48b7489285" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a71bf91b65544e8aa0ac47076223291"
I20260929 02:19:19.872445 15875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca8a4b243ff4446db4c3fd48b7489285" candidate_uuid: "91a00edef7674f4e9b6c061df3057f78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c14e13a625a144cbb95b88568976c6b3"
I20260929 02:19:19.872622 15593 raft_consensus.cc:3037] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.872630 15875 raft_consensus.cc:3037] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:19.875370 15593 raft_consensus.cc:2454] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 1.
I20260929 02:19:19.875370 15875 raft_consensus.cc:2454] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91a00edef7674f4e9b6c061df3057f78 in term 1.
I20260929 02:19:19.875758 15664 leader_election.cc:302] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 91a00edef7674f4e9b6c061df3057f78, 9a71bf91b65544e8aa0ac47076223291, c14e13a625a144cbb95b88568976c6b3; no voters:
I20260929 02:19:19.876011 15956 raft_consensus.cc:2781] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:19.876391 15956 raft_consensus.cc:686] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 1 LEADER]: Becoming Leader. State: Replica: 91a00edef7674f4e9b6c061df3057f78, State: Running, Role: LEADER
I20260929 02:19:19.876637 15956 consensus_queue.cc:237] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } }
I20260929 02:19:19.879014 15435 catalog_manager.cc:5495] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "91a00edef7674f4e9b6c061df3057f78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91a00edef7674f4e9b6c061df3057f78" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 33105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 } health_report { overall_health: UNKNOWN } } }
I20260929 02:19:19.900063 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.895434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.7.170.2:33105)
0929 02:19:19.895922 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.895927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.895945 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:19:19.895993 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.895996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.896056 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.896132 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.896430 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.897265 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.898474 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.898914 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.899004 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.899390 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.899490 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.899553 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.899632 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.899688 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":251,"threads_started":1}
I20260929 02:19:19.913444 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.908543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54713 (local address 127.7.170.1:39021)
0929 02:19:19.908988 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.908992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.909144 (+ 152us) server_negotiation.cc:407] Connection header received
0929 02:19:19.909307 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.909316 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.909378 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.909448 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.909846 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.910544 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.911878 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.912284 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.912411 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.912809 (+ 398us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.912918 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.913031 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.913135 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.913198 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":214,"threads_started":1}
I20260929 02:19:19.913463 16051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.908447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39021 (local address 127.7.170.2:54713)
0929 02:19:19.909071 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.909087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.909197 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.909461 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.909464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.909472 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:19:19.909704 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.909711 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.910638 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.910642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.911719 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.911731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.912438 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.912442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.912563 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.912614 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.913039 (+ 425us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.913048 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.913112 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.913210 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":286,"threads_started":1}
I20260929 02:19:19.920454 15875 raft_consensus.cc:1261] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Refusing update from remote peer 91a00edef7674f4e9b6c061df3057f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.920567 15593 raft_consensus.cc:1261] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Refusing update from remote peer 91a00edef7674f4e9b6c061df3057f78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:19.920959 16043 consensus_queue.cc:1035] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c14e13a625a144cbb95b88568976c6b3" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 44503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.921298 15956 consensus_queue.cc:1035] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a71bf91b65544e8aa0ac47076223291" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:19.943408 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.938509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:36923 (local address 127.7.170.2:33105)
0929 02:19:19.938697 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.938705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.939170 (+ 465us) server_negotiation.cc:407] Connection header received
0929 02:19:19.939335 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.939340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.939419 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.939492 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:19.939810 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.940509 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.941869 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.942286 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.942391 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.942791 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.942890 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.942960 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.943077 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.943148 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:19:19.943485 16057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.938499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:33105 (local address 127.7.170.1:36923)
0929 02:19:19.939076 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:19.939091 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:19.939219 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:19.939506 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:19.939509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:19.939519 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0929 02:19:19.939722 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.939729 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.940612 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.940616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.941748 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:19.941759 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.942441 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.942447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:19.942612 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.942648 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0929 02:19:19.942975 (+ 327us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0929 02:19:19.942986 (+ 11us) client_negotiation.cc:769] Sending connection context
0929 02:19:19.943050 (+ 64us) client_negotiation.cc:240] Negotiation successful
0929 02:19:19.943175 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":341,"threads_started":1}
I20260929 02:19:19.964367 15435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.964573 15435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:19.967041 15435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c77415acec1439da908774dac0871e8: Sending DeleteTablet for 3 replicas of tablet ca8a4b243ff4446db4c3fd48b7489285
I20260929 02:19:19.967700 15573 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:42900
I20260929 02:19:19.967698 15712 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:47480
I20260929 02:19:19.967799 15851 tablet_service.cc:1514] Processing DeleteTablet for tablet ca8a4b243ff4446db4c3fd48b7489285 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:19 UTC) from {username='oryx', principal='oryx/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45120
I20260929 02:19:19.967972 16017 tablet_replica.cc:330] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: stopping tablet replica
I20260929 02:19:19.968004 16019 tablet_replica.cc:330] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: stopping tablet replica
I20260929 02:19:19.968026 16018 tablet_replica.cc:330] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: stopping tablet replica
I20260929 02:19:19.968156 16017 raft_consensus.cc:2229] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:19:19.968164 16018 raft_consensus.cc:2229] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.968170 16019 raft_consensus.cc:2229] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:19.968336 16019 raft_consensus.cc:2258] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.968335 16018 raft_consensus.cc:2258] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.968436 16017 raft_consensus.cc:2258] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:19.968817 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 15507
I20260929 02:19:19.969429 16019 ts_tablet_manager.cc:1905] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.969472 16017 ts_tablet_manager.cc:1905] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.970184 16018 ts_tablet_manager.cc:1905] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:19.975521 16019 ts_tablet_manager.cc:1918] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.975668 16019 log.cc:1196] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/ca8a4b243ff4446db4c3fd48b7489285
I20260929 02:19:19.976082 16019 ts_tablet_manager.cc:1939] T ca8a4b243ff4446db4c3fd48b7489285 P 9a71bf91b65544e8aa0ac47076223291: Deleting consensus metadata
I20260929 02:19:19.976131 16018 ts_tablet_manager.cc:1918] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.976245 16018 log.cc:1196] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/ca8a4b243ff4446db4c3fd48b7489285
I20260929 02:19:19.976331 16017 ts_tablet_manager.cc:1918] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260929 02:19:19.976469 16017 log.cc:1196] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/ca8a4b243ff4446db4c3fd48b7489285
I20260929 02:19:19.976640 16018 ts_tablet_manager.cc:1939] T ca8a4b243ff4446db4c3fd48b7489285 P c14e13a625a144cbb95b88568976c6b3: Deleting consensus metadata
I20260929 02:19:19.977110 15422 catalog_manager.cc:4841] TS 9a71bf91b65544e8aa0ac47076223291 (127.7.170.1:39021): tablet ca8a4b243ff4446db4c3fd48b7489285 (table test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]) successfully deleted
I20260929 02:19:19.977344 16017 ts_tablet_manager.cc:1939] T ca8a4b243ff4446db4c3fd48b7489285 P 91a00edef7674f4e9b6c061df3057f78: Deleting consensus metadata
I20260929 02:19:19.977658 15420 catalog_manager.cc:4841] TS c14e13a625a144cbb95b88568976c6b3 (127.7.170.3:44503): tablet ca8a4b243ff4446db4c3fd48b7489285 (table test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]) successfully deleted
I20260929 02:19:19.978400 15419 catalog_manager.cc:4841] TS 91a00edef7674f4e9b6c061df3057f78 (127.7.170.2:33105): tablet ca8a4b243ff4446db4c3fd48b7489285 (table test-table [id=c4659cd0d42e4b6fa823e6fa876e13b7]) successfully deleted
Sep 29 02:19:19 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.1@KRBTEST.COM
I20260929 02:19:19.991240 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:19.971222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.7.170.1:39021)
0929 02:19:19.971375 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:19.971381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:19.978724 (+ 7343us) server_negotiation.cc:407] Connection header received
0929 02:19:19.978876 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:19.978882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:19.978943 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:19.979018 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:19.980123 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.980789 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.981964 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:19.982349 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:19.982459 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:19.986086 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:19.986102 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:19.986118 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:19.986148 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:19.988686 (+ 2538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:19.989354 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:19.989361 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:19.989363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:19.989474 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:19.989813 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:19.989816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:19.989818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:19.990150 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:19.990446 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:19.990751 (+ 305us) server_negotiation.cc:299] Negotiation successful
0929 02:19:19.990950 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:19:19.991808 15638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:20.137905 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 15507
I20260929 02:19:20.155651 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 15647
Sep 29 02:19:20 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.2@KRBTEST.COM
I20260929 02:19:20.169358 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:20.156348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.7.170.2:33105)
0929 02:19:20.156517 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:20.156523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:20.156588 (+ 65us) server_negotiation.cc:407] Connection header received
0929 02:19:20.156775 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:20.156781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:20.156851 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:20.156951 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:20.158093 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:20.158809 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:20.159986 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:20.160409 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:20.160515 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:20.164248 (+ 3733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:20.164261 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:20.164275 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:20.164312 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:20.166837 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:20.167444 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:20.167451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:20.167454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:20.167585 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:20.167901 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:20.167906 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:20.167908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:20.168236 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:20.168500 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:20.168802 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 02:19:20.169019 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:19:20.169919 15777 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:20.290446 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 15647
I20260929 02:19:20.309162 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 15786
Sep 29 02:19:20 dist-test-slave-8cs2 krb5kdc[15390](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.170.3@KRBTEST.COM
I20260929 02:19:20.323589 16062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:20.309858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.7.170.3:44503)
0929 02:19:20.310318 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:20.310322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:20.310337 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:19:20.310388 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:20.310391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:20.310446 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:20.310535 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:20.311820 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:20.312542 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:20.313818 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:20.314217 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:20.314325 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:20.318247 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:20.318262 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:20.318278 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:20.318321 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:20.320919 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:20.321664 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:20.321672 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:20.321674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:20.321806 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:20.322145 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:20.322149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:20.322151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:20.322495 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:20.322754 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:20.323052 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:19:20.323274 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":228,"threads_started":1}
I20260929 02:19:20.324193 15916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:20.442644 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 15786
I20260929 02:19:20.460673 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 15403
I20260929 02:19:20.461797 15476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:20.572351 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 15403
2026-09-29T02:19:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5719 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:20 dist-test-slave-8cs2 krb5kdc[16067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:20 dist-test-slave-8cs2 krb5kdc[16067](info): set up 2 sockets
Sep 29 02:19:20 dist-test-slave-8cs2 krb5kdc[16067](info): commencing operation
krb5kdc: starting...
W20260929 02:19:21.577921 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.966s user 0.006s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:21 dist-test-slave-8cs2 krb5kdc[16067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:19:21.781459 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41229
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:41229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:22.083204 16080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:22.083465 16080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:22.083587 16080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:22.090435 16080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:22.090518 16080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:22.090585 16080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:22.090656 16080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:22.090710 16080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:22.099931 16080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:41229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:22.101135 16080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:22.102356 16080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:22.113234 16085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:22.113274 16086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:22.113193 16088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:22.118250 16080 server_base.cc:1034] running on GCE node
I20260929 02:19:22.119009 16080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:22.120349 16080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:22.121541 16080 hybrid_clock.cc:648] HybridClock initialized: now 1790648362121498 us; error 41 us; skew 500 ppm
I20260929 02:19:22.121851 16080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-29T02:19:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1617 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:22 dist-test-slave-8cs2 krb5kdc[16094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:22 dist-test-slave-8cs2 krb5kdc[16094](info): set up 2 sockets
Sep 29 02:19:22 dist-test-slave-8cs2 krb5kdc[16094](info): commencing operation
krb5kdc: starting...
W20260929 02:19:23.193833 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.964s user 0.009s sys 0.011s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:23 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:19:23.399801 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40593
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:40593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:23.704463 16108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:23.704749 16108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:23.704873 16108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:23.712096 16108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:23.712204 16108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:23.712275 16108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:23.712353 16108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:23.712410 16108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:23.722069 16108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:40593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:23.723351 16108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:23.724643 16108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:23.735535 16113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:23.735483 16114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:23.735565 16116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:23.736359 16108 server_base.cc:1034] running on GCE node
I20260929 02:19:23.737161 16108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:23.738528 16108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:23.739754 16108 hybrid_clock.cc:648] HybridClock initialized: now 1790648363739690 us; error 49 us; skew 500 ppm
I20260929 02:19:23.740108 16108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:23 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:23.742974 16108 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:19:23.744319 16108 webserver.cc:460] Webserver started at http://127.7.170.62:39923/ using document root <none> and password file <none>
I20260929 02:19:23.744983 16108 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:23.745095 16108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:23.745388 16108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:23.748226 16108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "a9c1084cf4fb4c49893c23a081e88bcf"
format_stamp: "Formatted at 2026-09-29 02:19:23 on dist-test-slave-8cs2"
server_key: "4bcaff5f54f58089b549c3a32595bf13"
server_key_iv: "d6140e5ef2c72658ba702f39c953f64a"
server_key_version: "encryptionkey@0"
I20260929 02:19:23.748920 16108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "a9c1084cf4fb4c49893c23a081e88bcf"
format_stamp: "Formatted at 2026-09-29 02:19:23 on dist-test-slave-8cs2"
server_key: "4bcaff5f54f58089b549c3a32595bf13"
server_key_iv: "d6140e5ef2c72658ba702f39c953f64a"
server_key_version: "encryptionkey@0"
I20260929 02:19:23.753744 16108 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:19:23.756870 16123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:23.758131 16108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:19:23.758268 16108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "a9c1084cf4fb4c49893c23a081e88bcf"
format_stamp: "Formatted at 2026-09-29 02:19:23 on dist-test-slave-8cs2"
server_key: "4bcaff5f54f58089b549c3a32595bf13"
server_key_iv: "d6140e5ef2c72658ba702f39c953f64a"
server_key_version: "encryptionkey@0"
I20260929 02:19:23.758445 16108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:23.773129 16108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:23.777078 16108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:23.777333 16108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:23.792619 16108 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:40593
I20260929 02:19:23.792645 16174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:40593 every 8 connection(s)
I20260929 02:19:23.793896 16108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:23.797132 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16108
I20260929 02:19:23.797333 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:23.797736 16175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:23.798148 7848 external_mini_cluster.cc:1428] Setting key 61e0d5757edfaaa39f63e9890fbf9539
I20260929 02:19:23.811271 16175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Bootstrap starting.
Sep 29 02:19:23 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:23.814289 16175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:23.815526 16175 log.cc:824] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:23.818688 16175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: No bootstrap required, opened a new log
I20260929 02:19:23.820750 16178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:23.800110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.7.170.62:40593)
0929 02:19:23.800855 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:23.800866 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:23.800909 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:19:23.802528 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:23.802549 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:23.802929 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:23.803242 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:23.804892 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:23.806178 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:23.807864 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:23.808549 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:23.808679 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:23.812954 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:23.812965 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:23.812986 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:23.813026 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:23.816345 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:23.817110 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:23.817118 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:23.817126 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:23.817317 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:23.817708 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:23.817713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:23.817716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:23.818389 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:23.818648 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:23.819056 (+ 408us) server_negotiation.cc:299] Negotiation successful
0929 02:19:23.819483 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":338,"threads_started":1}
I20260929 02:19:23.826864 16175 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:23.827188 16175 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:23.827255 16175 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c1084cf4fb4c49893c23a081e88bcf, State: Initialized, Role: FOLLOWER
I20260929 02:19:23.827749 16175 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:23.827943 16175 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:23.828039 16175 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:23.828147 16175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:23.830338 16175 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:23.830766 16175 leader_election.cc:302] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c1084cf4fb4c49893c23a081e88bcf; no voters:
I20260929 02:19:23.831276 16175 leader_election.cc:288] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:23.831398 16180 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:23.832042 16180 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 LEADER]: Becoming Leader. State: Replica: a9c1084cf4fb4c49893c23a081e88bcf, State: Running, Role: LEADER
I20260929 02:19:23.832542 16180 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:23.833199 16175 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:23.835197 16181 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } } }
I20260929 02:19:23.835587 16181 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:23.835911 16182 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c1084cf4fb4c49893c23a081e88bcf. Latest consensus state: current_term: 1 leader_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } } }
I20260929 02:19:23.836231 16182 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:23.841331 16187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:23.846769 16187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:23.854875 16187 catalog_manager.cc:1347] Generated new cluster ID: 3209fe56fc3d4e96b12c7eb99ee03293
I20260929 02:19:23.854987 16187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:23.869717 16187 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:23.871131 16187 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:23.878500 16187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Generated new TSK 0
I20260929 02:19:23.879563 16187 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:19:23.929001 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:24.245442 16201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:24.245739 16201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:24.245893 16201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:24.253098 16201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:24.253219 16201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:24.253371 16201 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:24.263118 16201 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:24.264365 16201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:24.265668 16201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:24.278012 16206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:24.277985 16207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:24.277951 16209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:24.278865 16201 server_base.cc:1034] running on GCE node
I20260929 02:19:24.279474 16201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:24.280349 16201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:24.281546 16201 hybrid_clock.cc:648] HybridClock initialized: now 1790648364281498 us; error 34 us; skew 500 ppm
I20260929 02:19:24.281836 16201 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:24 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:24.284780 16201 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:19:24.286229 16201 webserver.cc:460] Webserver started at http://127.7.170.1:39599/ using document root <none> and password file <none>
I20260929 02:19:24.286896 16201 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:24.287006 16201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:24.287290 16201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:24.290086 16201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "87030a3fb024437cb6b50ad820f498a7"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "c0d98842fcf319021d7afb9df24a0b75"
server_key_iv: "4924c96c61450464e1ffd108e1f728c1"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.290796 16201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "87030a3fb024437cb6b50ad820f498a7"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "c0d98842fcf319021d7afb9df24a0b75"
server_key_iv: "4924c96c61450464e1ffd108e1f728c1"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.295641 16201 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:19:24.298949 16216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.300343 16201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:24.300493 16201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "87030a3fb024437cb6b50ad820f498a7"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "c0d98842fcf319021d7afb9df24a0b75"
server_key_iv: "4924c96c61450464e1ffd108e1f728c1"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.300662 16201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:24.326334 16201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:24.330472 16201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:24.330732 16201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:24.331663 16201 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:24.333093 16201 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:24.333171 16201 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.333292 16201 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:24.333353 16201 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.363353 16201 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:34071
I20260929 02:19:24.363399 16328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:34071 every 8 connection(s)
I20260929 02:19:24.364594 16201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:24.373051 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16201
I20260929 02:19:24.373224 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:24.373823 7848 external_mini_cluster.cc:1428] Setting key eaf3a268d6d933283750d1b7d860215f
Sep 29 02:19:24 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:24.384050 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:24.366987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:48319 (local address 127.7.170.62:40593)
0929 02:19:24.367429 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:24.367434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:24.368419 (+ 985us) server_negotiation.cc:407] Connection header received
0929 02:19:24.370222 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:24.370227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:24.370304 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:24.370405 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:24.371840 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.372558 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.374182 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.374625 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.374743 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:24.378352 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:24.378358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:24.378363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:24.378398 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:24.381230 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:24.381945 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:24.381950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:24.381953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:24.382085 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:24.382467 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:24.382471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:24.382474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:24.382788 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:24.382931 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:24.383627 (+ 696us) server_negotiation.cc:299] Negotiation successful
0929 02:19:24.383804 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":222,"threads_started":1}
I20260929 02:19:24.385449 16332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:24.367578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:48319)
0929 02:19:24.368259 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:24.368300 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:24.369972 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:24.370634 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:24.370649 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:24.371131 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:24.371685 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:24.371702 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.372688 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.372692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:24.374050 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:24.374062 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.374841 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.374884 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:24.375174 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:24.375893 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:24.375916 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:24.378112 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:24.381356 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:24.381363 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:24.381376 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:24.381788 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:24.382182 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:24.382185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:24.382188 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:24.382366 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:24.382926 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:24.382932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:24.383249 (+ 317us) client_negotiation.cc:769] Sending connection context
0929 02:19:24.383540 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 02:19:24.384086 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":271,"threads_started":1}
I20260929 02:19:24.387063 16329 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40593
I20260929 02:19:24.387341 16329 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:24.388022 16329 heartbeater.cc:511] Master 127.7.170.62:40593 requested a full tablet report, sending...
I20260929 02:19:24.389936 16140 ts_manager.cc:194] Registered new tserver with Master: 87030a3fb024437cb6b50ad820f498a7 (127.7.170.1:34071)
I20260929 02:19:24.391320 16140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:48319
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:19:24.423429 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:24.733716 16335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:24.734094 16335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:24.734248 16335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:24.742951 16335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:24.743065 16335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:24.743228 16335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:24.753307 16335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:24.754542 16335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:24.755786 16335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:24.768226 16340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:24.768288 16343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:24.768241 16341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:24.769030 16335 server_base.cc:1034] running on GCE node
I20260929 02:19:24.769620 16335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:24.770556 16335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:24.771756 16335 hybrid_clock.cc:648] HybridClock initialized: now 1790648364771703 us; error 47 us; skew 500 ppm
I20260929 02:19:24.772084 16335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:24 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:24.781787 16335 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:19:24.783290 16335 webserver.cc:460] Webserver started at http://127.7.170.2:42647/ using document root <none> and password file <none>
I20260929 02:19:24.783957 16335 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:24.784075 16335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:24.784375 16335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:24.787307 16335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "a01edf4684cf465b96ad49db04d37302"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "2d920b3c9bc25204df7757499f62e9fb"
server_key_iv: "268a654355ee1dd0d61a3848cc426364"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.788049 16335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "a01edf4684cf465b96ad49db04d37302"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "2d920b3c9bc25204df7757499f62e9fb"
server_key_iv: "268a654355ee1dd0d61a3848cc426364"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.793205 16335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:19:24.797310 16350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.798769 16335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:19:24.798919 16335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "a01edf4684cf465b96ad49db04d37302"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "2d920b3c9bc25204df7757499f62e9fb"
server_key_iv: "268a654355ee1dd0d61a3848cc426364"
server_key_version: "encryptionkey@0"
I20260929 02:19:24.799115 16335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:24.820449 16335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:24.824409 16335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:24.824643 16335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:24.825577 16335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:24.826861 16335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:24.826932 16335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.827014 16335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:24.827054 16335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:24.855211 16335 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:35181
I20260929 02:19:24.855253 16462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:35181 every 8 connection(s)
I20260929 02:19:24.856454 16335 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:24.863368 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16335
I20260929 02:19:24.863543 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:24.864035 7848 external_mini_cluster.cc:1428] Setting key 07b82116b1e8782ef55d7d63b548c3d1
Sep 29 02:19:24 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648364, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:24.876708 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:24.858841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:58783 (local address 127.7.170.62:40593)
0929 02:19:24.859047 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:24.859053 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:24.860220 (+ 1167us) server_negotiation.cc:407] Connection header received
0929 02:19:24.862001 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:24.862006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:24.862073 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:24.862150 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:24.863849 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.864690 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.866598 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.866970 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.867068 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:24.870990 (+ 3922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:24.870995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:24.870998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:24.871029 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:24.873487 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:24.874352 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:24.874357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:24.874359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:24.874508 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:24.874926 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:24.874931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:24.874934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:24.875278 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:24.875418 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:24.876187 (+ 769us) server_negotiation.cc:299] Negotiation successful
0929 02:19:24.876409 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:19:24.877578 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:24.859390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:58783)
0929 02:19:24.860067 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:24.860105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:24.861747 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:24.862371 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:24.862379 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:24.862876 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:24.863650 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:24.863675 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.864839 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.864845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:24.866450 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:24.866465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:24.867193 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:24.867201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:24.867496 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:24.868163 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:24.868194 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:24.870755 (+ 2561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:24.873626 (+ 2871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:24.873635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:24.873654 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:24.874209 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:24.874603 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:24.874607 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:24.874610 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:24.874798 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:24.875429 (+ 631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:24.875436 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:24.875772 (+ 336us) client_negotiation.cc:769] Sending connection context
0929 02:19:24.876049 (+ 277us) client_negotiation.cc:240] Negotiation successful
0929 02:19:24.876442 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":249,"threads_started":1}
I20260929 02:19:24.879096 16463 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40593
I20260929 02:19:24.879369 16463 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:24.880231 16463 heartbeater.cc:511] Master 127.7.170.62:40593 requested a full tablet report, sending...
I20260929 02:19:24.881687 16140 ts_manager.cc:194] Registered new tserver with Master: a01edf4684cf465b96ad49db04d37302 (127.7.170.2:35181)
I20260929 02:19:24.882405 16140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:58783
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:19:24.912093 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:25.228616 16468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:25.228902 16468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:25.229065 16468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:25.236043 16468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:25.236150 16468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:25.236300 16468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:25.245538 16468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:25.246773 16468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:25.248028 16468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:25.260130 16474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:25.260186 16473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:25.260376 16476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:25.260820 16468 server_base.cc:1034] running on GCE node
I20260929 02:19:25.261395 16468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:25.262282 16468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:25.263489 16468 hybrid_clock.cc:648] HybridClock initialized: now 1790648365263419 us; error 59 us; skew 500 ppm
I20260929 02:19:25.263762 16468 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:25 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:25.266588 16468 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:19:25.267918 16468 webserver.cc:460] Webserver started at http://127.7.170.3:35839/ using document root <none> and password file <none>
I20260929 02:19:25.268554 16468 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:25.268667 16468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:25.268980 16468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:25.271699 16468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "67622e4652d94c0b83aa7c7cdbcb2407"
format_stamp: "Formatted at 2026-09-29 02:19:25 on dist-test-slave-8cs2"
server_key: "86c38b4913972faccabe257e1d449a64"
server_key_iv: "fde54a7504a364323742acc1a4a46ea7"
server_key_version: "encryptionkey@0"
I20260929 02:19:25.272413 16468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "67622e4652d94c0b83aa7c7cdbcb2407"
format_stamp: "Formatted at 2026-09-29 02:19:25 on dist-test-slave-8cs2"
server_key: "86c38b4913972faccabe257e1d449a64"
server_key_iv: "fde54a7504a364323742acc1a4a46ea7"
server_key_version: "encryptionkey@0"
I20260929 02:19:25.277320 16468 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:19:25.280568 16483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:25.281970 16468 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:25.282116 16468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "67622e4652d94c0b83aa7c7cdbcb2407"
format_stamp: "Formatted at 2026-09-29 02:19:25 on dist-test-slave-8cs2"
server_key: "86c38b4913972faccabe257e1d449a64"
server_key_iv: "fde54a7504a364323742acc1a4a46ea7"
server_key_version: "encryptionkey@0"
I20260929 02:19:25.282277 16468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:25.302120 16468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:25.305965 16468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:25.306193 16468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:25.307027 16468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:25.308213 16468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:25.308285 16468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:25.308373 16468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:25.308413 16468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:25.336345 16468 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:45623
I20260929 02:19:25.336388 16596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:45623 every 8 connection(s)
I20260929 02:19:25.337543 16468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:25.341348 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16468
I20260929 02:19:25.341513 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:25.342036 7848 external_mini_cluster.cc:1428] Setting key ace9a16339bd0586e0940f54376eb04e
Sep 29 02:19:25 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:25.355693 16331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:25.339871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:45271 (local address 127.7.170.62:40593)
0929 02:19:25.340030 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:25.340035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:25.341251 (+ 1216us) server_negotiation.cc:407] Connection header received
0929 02:19:25.343389 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:25.343392 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:25.343489 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:25.343564 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:25.345082 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.345849 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.347381 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.347789 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.347875 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:25.351023 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:25.351028 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:25.351031 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:25.351062 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:25.353282 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.353888 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.353891 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.353894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.354001 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.354304 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.354307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.354309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.354548 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:25.354647 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:25.355222 (+ 575us) server_negotiation.cc:299] Negotiation successful
0929 02:19:25.355420 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:19:25.356712 16599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:25.340356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:45271)
0929 02:19:25.341068 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:25.341131 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:25.343135 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:25.343797 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:25.343805 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:25.344278 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:25.344902 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:25.344950 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.345970 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.345974 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:25.347265 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:25.347276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.347948 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.347953 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:25.348142 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:25.348697 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:25.348722 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:25.350829 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:25.353400 (+ 2571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:25.353407 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:25.353419 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:25.353799 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:25.354071 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:25.354074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:25.354076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:25.354230 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:25.354630 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:25.354636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:25.354907 (+ 271us) client_negotiation.cc:769] Sending connection context
0929 02:19:25.355170 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 02:19:25.355622 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":252,"threads_started":1}
I20260929 02:19:25.357873 16597 heartbeater.cc:347] Connected to a master server at 127.7.170.62:40593
I20260929 02:19:25.358146 16597 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:25.358807 16597 heartbeater.cc:511] Master 127.7.170.62:40593 requested a full tablet report, sending...
I20260929 02:19:25.360067 16140 ts_manager.cc:194] Registered new tserver with Master: 67622e4652d94c0b83aa7c7cdbcb2407 (127.7.170.3:45623)
I20260929 02:19:25.360793 16140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:45271
I20260929 02:19:25.369025 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.1@KRBTEST.COM" created.
I20260929 02:19:25.394238 16329 heartbeater.cc:503] Master 127.7.170.62:40593 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
Entry for principal oryx/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab.
WARNING: no policy specified for oryx/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
Entry for principal oryx/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab.
WARNING: no policy specified for oryx/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
Entry for principal oryx/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab.
I20260929 02:19:25.488588 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16201
Sep 29 02:19:25 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:19:25.503433 16607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:25.489732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50290 (local address 127.7.170.1:34071)
0929 02:19:25.490367 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:25.490374 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:25.490401 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:19:25.490460 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:25.490466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:25.490682 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:25.490810 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:25.492038 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.493145 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.494321 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.494911 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.495005 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:25.498588 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:25.498599 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:25.498613 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:25.498648 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:25.501128 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.501770 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.501777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.501779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.501901 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.502150 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.502153 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.502155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.502460 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:25.502674 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:25.502874 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:19:25.503075 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":332,"threads_started":1}
I20260929 02:19:25.504760 16324 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:25.585130 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16201
I20260929 02:19:25.598479 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16335
Sep 29 02:19:25 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:19:25.612668 16609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:25.599225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.7.170.2:35181)
0929 02:19:25.599700 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:25.599706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:25.599732 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:19:25.599788 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:25.599795 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:25.600012 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:25.600140 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:25.601239 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.602285 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.603413 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.604019 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.604133 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:25.607872 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:25.607884 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:25.607897 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:25.607927 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:25.610555 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.611074 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.611080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.611082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.611188 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.611429 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.611432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.611434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.611733 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:25.611948 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:25.612143 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 02:19:25.612315 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":241,"threads_started":1}
I20260929 02:19:25.613878 16458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:25.693415 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16335
I20260929 02:19:25.705914 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16468
Sep 29 02:19:25 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:19:25.721076 16611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:25.706833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57134 (local address 127.7.170.3:45623)
0929 02:19:25.707447 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:25.707454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:25.707482 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:19:25.707538 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:25.707544 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:25.707758 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:25.707910 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:25.709188 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.710198 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.711389 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:25.711926 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:25.712041 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:25.716109 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:25.716119 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:25.716133 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:25.716167 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:25.718554 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.719128 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.719134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.719137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.719259 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:25.719590 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:25.719593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:25.719595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:25.719913 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:25.720200 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:25.720468 (+ 268us) server_negotiation.cc:299] Negotiation successful
0929 02:19:25.720711 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":323,"threads_started":1}
I20260929 02:19:25.722389 16592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:25.807159 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16468
I20260929 02:19:25.819715 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16108
I20260929 02:19:25.820799 16170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:25.921096 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16108
I20260929 02:19:25.938153 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40593
--webserver_interface=127.7.170.62
--webserver_port=39923
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:40593
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:26.243312 16614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:26.243583 16614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:26.243706 16614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:26.250708 16614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:26.250808 16614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:26.250876 16614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:26.250958 16614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:26.251011 16614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:26.260432 16614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:40593
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:40593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=39923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:26.261689 16614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:26.262961 16614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:26.273680 16619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:26.273649 16620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:26.273729 16622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:26.274487 16614 server_base.cc:1034] running on GCE node
I20260929 02:19:26.275225 16614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:26.276540 16614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:26.277730 16614 hybrid_clock.cc:648] HybridClock initialized: now 1790648366277687 us; error 30 us; skew 500 ppm
I20260929 02:19:26.278044 16614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648366, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:26.280898 16614 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:19:26.282302 16614 webserver.cc:460] Webserver started at http://127.7.170.62:39923/ using document root <none> and password file <none>
I20260929 02:19:26.282964 16614 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:26.283088 16614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:26.288293 16614 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:26.290984 16629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:26.292246 16614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:19:26.292385 16614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "a9c1084cf4fb4c49893c23a081e88bcf"
format_stamp: "Formatted at 2026-09-29 02:19:23 on dist-test-slave-8cs2"
server_key: "4bcaff5f54f58089b549c3a32595bf13"
server_key_iv: "d6140e5ef2c72658ba702f39c953f64a"
server_key_version: "encryptionkey@0"
I20260929 02:19:26.293329 16614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:26.307219 16614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:26.311190 16614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:26.311421 16614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:26.327365 16614 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:40593
I20260929 02:19:26.327383 16680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:40593 every 8 connection(s)
I20260929 02:19:26.328732 16614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:26.335201 16681 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:19:26.336521 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16614
I20260929 02:19:26.337451 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:34071
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:40593
--webserver_port=39599
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:19:26.339234 16681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Bootstrap starting.
I20260929 02:19:26.358558 16681 log.cc:824] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:26.366901 16681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:19:26.367354 16681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Bootstrap complete.
I20260929 02:19:26.375819 16681 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:26.376813 16681 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c1084cf4fb4c49893c23a081e88bcf, State: Initialized, Role: FOLLOWER
I20260929 02:19:26.377429 16681 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:26.377648 16681 raft_consensus.cc:388] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:26.377750 16681 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:26.377908 16681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:19:26.381460 16681 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:26.381893 16681 leader_election.cc:302] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a9c1084cf4fb4c49893c23a081e88bcf; no voters:
I20260929 02:19:26.382440 16681 leader_election.cc:288] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:19:26.382573 16685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:19:26.383711 16685 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [term 2 LEADER]: Becoming Leader. State: Replica: a9c1084cf4fb4c49893c23a081e88bcf, State: Running, Role: LEADER
I20260929 02:19:26.384264 16685 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } }
I20260929 02:19:26.384529 16681 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:26.386607 16686 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } } }
I20260929 02:19:26.387292 16686 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:26.387284 16687 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9c1084cf4fb4c49893c23a081e88bcf. Latest consensus state: current_term: 2 leader_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a9c1084cf4fb4c49893c23a081e88bcf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 40593 } } }
I20260929 02:19:26.387780 16687 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:26.389083 16691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:26.394856 16691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:26.396319 16691 catalog_manager.cc:1259] Loaded cluster ID: 3209fe56fc3d4e96b12c7eb99ee03293
I20260929 02:19:26.396457 16691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:26.400590 16691 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:26.402122 16691 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9c1084cf4fb4c49893c23a081e88bcf: Loaded TSK: 0
I20260929 02:19:26.403986 16691 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:19:26.684789 16683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:26.685113 16683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:26.685256 16683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:26.692457 16683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:26.692571 16683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:26.692719 16683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:26.702620 16683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.1.keytab
--principal=oryx/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:34071
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=39599
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:26.704038 16683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:26.705327 16683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:26.719502 16710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:26.719592 16708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:26.719815 16707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:26.720287 16683 server_base.cc:1034] running on GCE node
I20260929 02:19:26.721010 16683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:26.722182 16683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:26.723448 16683 hybrid_clock.cc:648] HybridClock initialized: now 1790648366723385 us; error 43 us; skew 500 ppm
I20260929 02:19:26.723804 16683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648366, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:26.727566 16683 init.cc:377] Logged in from keytab as oryx/127.7.170.1@KRBTEST.COM (short username oryx)
I20260929 02:19:26.729177 16683 webserver.cc:460] Webserver started at http://127.7.170.1:39599/ using document root <none> and password file <none>
I20260929 02:19:26.730008 16683 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:26.730190 16683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:26.736784 16683 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260929 02:19:26.740670 16717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:26.742177 16683 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 02:19:26.742331 16683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "87030a3fb024437cb6b50ad820f498a7"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "c0d98842fcf319021d7afb9df24a0b75"
server_key_iv: "4924c96c61450464e1ffd108e1f728c1"
server_key_version: "encryptionkey@0"
I20260929 02:19:26.743580 16683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:26.768229 16683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:26.772351 16683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:26.772594 16683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:26.773571 16683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:26.774827 16683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:26.774917 16683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:26.775030 16683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:26.775105 16683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:26.802366 16683 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:34071
I20260929 02:19:26.802407 16829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:34071 every 8 connection(s)
I20260929 02:19:26.803834 16683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:26.810063 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16683
I20260929 02:19:26.810642 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:35181
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:40593
--webserver_port=42647
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:26.820858 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:26.806652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:48665)
0929 02:19:26.807388 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:26.807422 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:26.809064 (+ 1642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:26.810456 (+ 1392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:26.810466 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:26.811057 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:26.811585 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.811602 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.813182 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.813186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.814442 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.814457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.815465 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.815472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.815725 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.816378 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:26.816413 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:26.820006 (+ 3593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":304,"threads_started":1}
W20260929 02:19:26.821506 16830 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:40593 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.170.62:40593: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
W20260929 02:19:26.821537 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:26.806611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:48665 (local address 127.7.170.62:40593)
0929 02:19:26.807309 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:26.807326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:26.807546 (+ 220us) server_negotiation.cc:407] Connection header received
0929 02:19:26.809583 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:26.809602 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:26.809934 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:26.810304 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:26.811762 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.813032 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.814587 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.815249 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.815412 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.820399 (+ 4987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:48665: BlockingRecv error: recv got EOF from 127.7.170.1:48665 (error 108)
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":231,"threads_started":1}
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:26.829578 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:26.822029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:55971)
0929 02:19:26.822257 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:26.822274 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:26.822364 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:26.822659 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:26.822662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:26.822930 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:26.823078 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.823084 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.824277 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.824280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.825180 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.825190 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.826041 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.826045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.826158 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.826639 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:26.826655 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:26.829320 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125}
W20260929 02:19:26.829617 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:26.822243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:55971 (local address 127.7.170.62:40593)
0929 02:19:26.822409 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:26.822414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:26.822431 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:26.822477 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:26.822482 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:26.822544 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:26.822630 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:26.823193 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.824169 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.825297 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.825907 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.826042 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.829467 (+ 3425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:55971: BlockingRecv error: recv got EOF from 127.7.170.1:55971 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:26 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:26.837271 16833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:26.830270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:42619)
0929 02:19:26.830408 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:26.830425 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:26.830527 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:26.830798 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:26.830803 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:26.831071 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:26.831197 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.831204 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.832117 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.832121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.833031 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:26.833042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.833648 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.833654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:26.833823 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.834295 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:26.834310 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:26.836995 (+ 2685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:19:26.837320 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:26.830360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:42619 (local address 127.7.170.62:40593)
0929 02:19:26.830497 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:26.830501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:26.830518 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:26.830624 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:26.830629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:26.830693 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:26.830766 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:26.831317 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.831989 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.833148 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:26.833529 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:26.833627 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:26.837144 (+ 3517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:42619: BlockingRecv error: recv got EOF from 127.7.170.1:42619 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 02:19:26.837522 16830 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:19:27.117535 16834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:27.117798 16834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:27.117921 16834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:27.124914 16834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:27.125041 16834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:27.125192 16834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:27.134371 16834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.2.keytab
--principal=oryx/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:35181
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=42647
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:27.135548 16834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:27.136783 16834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:27.148628 16840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:27.148665 16842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:27.148694 16839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:27.149447 16834 server_base.cc:1034] running on GCE node
I20260929 02:19:27.149999 16834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:27.150789 16834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:27.151968 16834 hybrid_clock.cc:648] HybridClock initialized: now 1790648367151927 us; error 34 us; skew 500 ppm
I20260929 02:19:27.152235 16834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648367, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:27.155012 16834 init.cc:377] Logged in from keytab as oryx/127.7.170.2@KRBTEST.COM (short username oryx)
I20260929 02:19:27.156291 16834 webserver.cc:460] Webserver started at http://127.7.170.2:42647/ using document root <none> and password file <none>
I20260929 02:19:27.156914 16834 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:27.157075 16834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:27.161886 16834 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260929 02:19:27.164271 16849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.165468 16834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:19:27.165616 16834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "a01edf4684cf465b96ad49db04d37302"
format_stamp: "Formatted at 2026-09-29 02:19:24 on dist-test-slave-8cs2"
server_key: "2d920b3c9bc25204df7757499f62e9fb"
server_key_iv: "268a654355ee1dd0d61a3848cc426364"
server_key_version: "encryptionkey@0"
I20260929 02:19:27.166499 16834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:27.184652 16834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:27.188215 16834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:27.188444 16834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:27.189329 16834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:27.190523 16834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:27.190620 16834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.190726 16834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:27.190800 16834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.219977 16834 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:35181
I20260929 02:19:27.220031 16961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:35181 every 8 connection(s)
I20260929 02:19:27.221421 16834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:27.230643 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16834
I20260929 02:19:27.231185 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:45623
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:40593
--webserver_port=35839
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43447
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:19:27.236016 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.223660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:53249 (local address 127.7.170.62:40593)
0929 02:19:27.223821 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.223825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.224913 (+ 1088us) server_negotiation.cc:407] Connection header received
0929 02:19:27.226691 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.226696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.226775 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.226859 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.228268 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.228993 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.230467 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.230910 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.231021 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.235826 (+ 4805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:53249: BlockingRecv error: recv got EOF from 127.7.170.2:53249 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:27.236840 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.224130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:53249)
0929 02:19:27.224776 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.224812 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.226433 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.227099 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.227107 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.227576 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.228107 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.228125 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.229145 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.229149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.230336 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.230347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.231116 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.231123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.231385 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.232002 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.232024 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.235676 (+ 3652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":243,"threads_started":1}
W20260929 02:19:27.237507 16962 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:40593 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.170.62:40593: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:27.244727 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.238005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:38969)
0929 02:19:27.238145 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.238162 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.238235 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.238490 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.238493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.238758 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.238880 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.238887 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.239804 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.239809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.240671 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.240681 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.241361 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.241367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.241523 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.242007 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.242023 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.244493 (+ 2470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260929 02:19:27.245250 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.238099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:38969 (local address 127.7.170.62:40593)
0929 02:19:27.238247 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.238253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.238269 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:27.238315 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.238320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.238383 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.238475 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.239011 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.239680 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.240796 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.241230 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.241340 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.244703 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:38969: BlockingRecv error: recv got EOF from 127.7.170.2:38969 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:27.252251 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.245356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:47365)
0929 02:19:27.245562 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.245578 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.245664 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.245922 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.245926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.246198 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.246324 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.246331 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.247211 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.247216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.248116 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.248126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.248816 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.248821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.249043 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.249564 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.249580 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.252000 (+ 2420us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94}
W20260929 02:19:27.252516 16962 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:19:27.252664 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.245515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:47365 (local address 127.7.170.62:40593)
0929 02:19:27.245672 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.245677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.245694 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:27.245751 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.245755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.245817 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.245888 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.246443 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.247096 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.248251 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.248694 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.248795 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.252204 (+ 3409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:47365: BlockingRecv error: recv got EOF from 127.7.170.2:47365 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260929 02:19:27.543066 16965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:27.543354 16965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:27.543486 16965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:27.550935 16965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:27.551052 16965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:27.551204 16965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:27.560920 16965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43447
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/krb5kdc/oryx_127.7.170.3.keytab
--principal=oryx/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:45623
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=35839
--tserver_master_addrs=127.7.170.62:40593
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:27.562214 16965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:27.563498 16965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:27.575776 16973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:27.575845 16970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:27.575872 16971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:27.576646 16965 server_base.cc:1034] running on GCE node
I20260929 02:19:27.577217 16965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:27.578047 16965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:27.579232 16965 hybrid_clock.cc:648] HybridClock initialized: now 1790648367579184 us; error 44 us; skew 500 ppm
I20260929 02:19:27.579517 16965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648367, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:27.582394 16965 init.cc:377] Logged in from keytab as oryx/127.7.170.3@KRBTEST.COM (short username oryx)
I20260929 02:19:27.583699 16965 webserver.cc:460] Webserver started at http://127.7.170.3:35839/ using document root <none> and password file <none>
I20260929 02:19:27.584321 16965 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:27.584435 16965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:27.589615 16965 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:19:27.592077 16980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.593302 16965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:27.593437 16965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "67622e4652d94c0b83aa7c7cdbcb2407"
format_stamp: "Formatted at 2026-09-29 02:19:25 on dist-test-slave-8cs2"
server_key: "86c38b4913972faccabe257e1d449a64"
server_key_iv: "fde54a7504a364323742acc1a4a46ea7"
server_key_version: "encryptionkey@0"
I20260929 02:19:27.594326 16965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:27.621913 16965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:27.626147 16965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:27.626394 16965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:27.627319 16965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:27.628645 16965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:27.628731 16965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.628829 16965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:27.628906 16965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:27.658906 16965 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:45623
I20260929 02:19:27.658959 17092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:45623 every 8 connection(s)
I20260929 02:19:27.660259 16965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:27.663213 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 16965
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
W20260929 02:19:27.677433 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.662668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:58389 (local address 127.7.170.62:40593)
0929 02:19:27.662857 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.662862 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.664813 (+ 1951us) server_negotiation.cc:407] Connection header received
0929 02:19:27.667128 (+ 2315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.667131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.667198 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.667311 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.668862 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.669668 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.671726 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.672131 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.672218 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.677181 (+ 4963us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:58389: BlockingRecv error: recv got EOF from 127.7.170.3:58389 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:19:27.678139 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.663149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:58389)
0929 02:19:27.664677 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.664713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.666849 (+ 2136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.667499 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.667508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.667972 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.668700 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.668724 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.669819 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.669823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.671582 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.671596 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.672345 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.672353 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.672643 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.673325 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.673346 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.676960 (+ 3614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1299,"thread_start_us":308,"threads_started":1}
W20260929 02:19:27.678747 17093 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:40593 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.170.62:40593: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
I20260929 02:19:27.679694 17097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.664056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.7.170.62:40593)
0929 02:19:27.665207 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.665211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.665225 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:19:27.665271 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.665275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.665329 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.665410 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.667214 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.668042 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.669606 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.670048 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.670130 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.673114 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:27.673126 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:27.673147 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:27.673197 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:27.676156 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:27.676779 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:27.676786 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:27.676793 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:27.677001 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:27.677317 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:27.677322 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:27.677326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:27.677827 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:27.678112 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:27.679123 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0929 02:19:27.679416 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1073,"thread_start_us":981,"threads_started":1}
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:27.687040 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.679269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:43937)
0929 02:19:27.679432 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.679449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.679557 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.679958 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.679963 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.680339 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.680505 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.680516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.681818 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.681821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.682947 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.682958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.683717 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.683721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.683840 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.684317 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.684333 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.686839 (+ 2506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260929 02:19:27.687207 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.679362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:43937 (local address 127.7.170.62:40593)
0929 02:19:27.679522 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.679527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.679543 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:27.679662 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.679668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.679731 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.679801 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.680644 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.681701 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.683077 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.683600 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.683691 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.687020 (+ 3329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:43937: BlockingRecv error: recv got EOF from 127.7.170.3:43937 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:27.694422 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.687613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:54583)
0929 02:19:27.687752 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.687766 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.687854 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.688135 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.688140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.688412 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.688550 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.688557 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.689582 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.689587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.690469 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.690479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.691069 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.691074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.691233 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.691682 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.691698 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.694154 (+ 2456us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:19:27.694667 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.687699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:54583 (local address 127.7.170.62:40593)
0929 02:19:27.687919 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.687924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.687939 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:19:27.687976 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.687979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.688028 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.688102 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.688675 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.689444 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.690594 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.690949 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.691048 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.694343 (+ 3295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:54583: BlockingRecv error: recv got EOF from 127.7.170.3:54583 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260929 02:19:27.694703 17093 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:27 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:27.845580 17098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:27.838024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:38111)
0929 02:19:27.838487 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:27.838502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:27.838610 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:27.838851 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:27.838853 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:27.839276 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:27.839455 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.839465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.840363 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.840367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.841365 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:27.841377 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.842013 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.842017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:27.842126 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.842658 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:27.842680 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:27.845264 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
W20260929 02:19:27.845661 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:27.838089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:38111 (local address 127.7.170.62:40593)
0929 02:19:27.838247 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:27.838252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:27.838563 (+ 311us) server_negotiation.cc:407] Connection header received
0929 02:19:27.838691 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:27.838695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:27.838751 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:27.838825 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:27.839561 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.840250 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.841471 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:27.841877 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:27.841964 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:27.845462 (+ 3498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:38111: BlockingRecv error: recv got EOF from 127.7.170.1:38111 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:28.261155 17099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:28.253127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:53745)
0929 02:19:28.253555 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:28.253571 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:28.253681 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:28.254029 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:28.254033 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:28.254445 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:28.254635 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.254643 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.255627 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.255632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.256603 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.256614 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.257471 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.257475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.257596 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.258200 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:28.258224 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:28.260829 (+ 2605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":227,"threads_started":1}
W20260929 02:19:28.261315 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:28.253184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:53745 (local address 127.7.170.62:40593)
0929 02:19:28.253396 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:28.253402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:28.253650 (+ 248us) server_negotiation.cc:407] Connection header received
0929 02:19:28.253774 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:28.253778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:28.253864 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:28.253946 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:28.254770 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.255500 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.256754 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.257323 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.257435 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.261093 (+ 3658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:53745: BlockingRecv error: recv got EOF from 127.7.170.2:53745 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:28.703205 17100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:28.695286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:50083)
0929 02:19:28.695807 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:28.695823 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:28.695938 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:28.696215 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:28.696219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:28.696680 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:28.696898 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.696909 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.698008 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.698014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.699055 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.699068 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.699683 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.699688 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.699806 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.700316 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:28.700332 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:28.702987 (+ 2655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":282,"threads_started":1}
W20260929 02:19:28.703419 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:28.695333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:50083 (local address 127.7.170.62:40593)
0929 02:19:28.695497 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:28.695503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:28.695921 (+ 418us) server_negotiation.cc:407] Connection header received
0929 02:19:28.696021 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:28.696025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:28.696086 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:28.696164 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:28.697053 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.697836 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.699166 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.699545 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.699630 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.703203 (+ 3573us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:50083: BlockingRecv error: recv got EOF from 127.7.170.3:50083 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:28 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:28.854161 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:28.846471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:37187)
0929 02:19:28.846931 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:28.846947 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:28.847070 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:28.847366 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:28.847370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:28.847744 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:28.847884 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.847892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.848904 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.848908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.849951 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:28.849964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.850572 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.850576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:28.850679 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.851192 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:28.851209 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:28.853840 (+ 2631us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":245,"threads_started":1}
W20260929 02:19:28.854288 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:28.846548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:37187 (local address 127.7.170.62:40593)
0929 02:19:28.846756 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:28.846762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:28.847019 (+ 257us) server_negotiation.cc:407] Connection header received
0929 02:19:28.847175 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:28.847178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:28.847238 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:28.847304 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:28.848029 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.848780 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.850066 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:28.850453 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:28.850550 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:28.854051 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:37187: BlockingRecv error: recv got EOF from 127.7.170.1:37187 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:29.269368 17102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:29.261960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:41575)
0929 02:19:29.262467 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:29.262505 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:29.262649 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:29.262987 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:29.262993 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:29.263469 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:29.263648 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.263657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.264602 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.264606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.265525 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.265537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.266153 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.266158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.266263 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.266754 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:29.266771 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:29.269135 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":260,"threads_started":1}
W20260929 02:19:29.269547 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:29.262033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:41575 (local address 127.7.170.62:40593)
0929 02:19:29.262198 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:29.262204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:29.262570 (+ 366us) server_negotiation.cc:407] Connection header received
0929 02:19:29.262784 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:29.262788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:29.262849 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:29.262921 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:29.263777 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.264479 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.265631 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.266044 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.266131 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.269335 (+ 3204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:41575: BlockingRecv error: recv got EOF from 127.7.170.2:41575 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:29.711683 17103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:29.704036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:34467)
0929 02:19:29.704532 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:29.704549 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:29.704671 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:29.704963 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:29.704966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:29.705344 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:29.705501 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.705510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.706457 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.706461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.707368 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.707379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.707994 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.707998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.708103 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.708735 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:29.708753 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:29.711448 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":267,"threads_started":1}
W20260929 02:19:29.711865 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:29.704182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:34467 (local address 127.7.170.62:40593)
0929 02:19:29.704331 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:29.704336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:29.704611 (+ 275us) server_negotiation.cc:407] Connection header received
0929 02:19:29.704761 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:29.704765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:29.704826 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:29.704901 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:29.705614 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.706343 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.707474 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.707884 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.707971 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.711620 (+ 3649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:34467: BlockingRecv error: recv got EOF from 127.7.170.3:34467 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:29 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:29.866678 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:29.855049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:58011)
0929 02:19:29.855543 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:29.855563 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:29.855673 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:29.855923 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:29.855926 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:29.856229 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:29.856399 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.856407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.857351 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.857354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.858243 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:29.858254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.858794 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.858798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:29.858923 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.863467 (+ 4544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:29.863499 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:29.866329 (+ 2830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":255,"threads_started":1}
W20260929 02:19:29.866768 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:29.855146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:58011 (local address 127.7.170.62:40593)
0929 02:19:29.855317 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:29.855321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:29.855625 (+ 304us) server_negotiation.cc:407] Connection header received
0929 02:19:29.855762 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:29.855765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:29.855823 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:29.855892 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:29.856519 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.857243 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.858345 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:29.858696 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:29.858778 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:29.866541 (+ 7763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:58011: BlockingRecv error: recv got EOF from 127.7.170.1:58011 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:30.277954 17105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:30.270210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:43861)
0929 02:19:30.270696 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:30.270715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:30.270896 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:30.271206 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:30.271211 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:30.271632 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:30.271835 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.271846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.272806 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.272811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.273732 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.273742 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.274377 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.274383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.274558 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.275124 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:30.275149 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:30.277712 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":242,"threads_started":1}
W20260929 02:19:30.278103 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:30.270268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:43861 (local address 127.7.170.62:40593)
0929 02:19:30.270452 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:30.270459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:30.270791 (+ 332us) server_negotiation.cc:407] Connection header received
0929 02:19:30.271021 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:30.271024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:30.271083 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:30.271159 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:30.271965 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.272683 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.273867 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.274243 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.274392 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.277905 (+ 3513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:43861: BlockingRecv error: recv got EOF from 127.7.170.2:43861 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:30.720593 17106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:30.712709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:51481)
0929 02:19:30.713285 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:30.713301 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:30.713434 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:30.713716 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:30.713719 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:30.714039 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:30.714202 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.714211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.715144 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.715148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.716041 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.716054 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.716727 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.716732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.716895 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.717636 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:30.717661 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:30.720356 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":357,"threads_started":1}
W20260929 02:19:30.720700 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:30.712859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:51481 (local address 127.7.170.62:40593)
0929 02:19:30.713051 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:30.713056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:30.713384 (+ 328us) server_negotiation.cc:407] Connection header received
0929 02:19:30.713527 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:30.713531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:30.713606 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:30.713688 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:30.714308 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.715005 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.716181 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.716586 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.716688 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.720535 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:51481: BlockingRecv error: recv got EOF from 127.7.170.3:51481 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:30 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:30.875453 17107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:30.867591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:60389)
0929 02:19:30.868087 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:30.868107 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:30.868241 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:30.868577 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:30.868582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:30.869087 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:30.869233 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.869242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.870224 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.870229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.871183 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:30.871195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.871831 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.871835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:30.871956 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.872490 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:30.872507 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:30.875137 (+ 2630us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
W20260929 02:19:30.875573 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:30.867687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:60389 (local address 127.7.170.62:40593)
0929 02:19:30.867916 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:30.867922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:30.868208 (+ 286us) server_negotiation.cc:407] Connection header received
0929 02:19:30.868354 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:30.868360 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:30.868441 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:30.868536 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:30.869362 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.870095 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.871325 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:30.871720 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:30.871821 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:30.875345 (+ 3524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:60389: BlockingRecv error: recv got EOF from 127.7.170.1:60389 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:31.286084 17108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:31.278727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:45329)
0929 02:19:31.279138 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:31.279154 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:31.279262 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:31.279515 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:31.279518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:31.279837 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:31.279979 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.280000 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.280864 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.280867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.281820 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.281830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.282414 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.282418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.282535 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.283079 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:31.283096 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:31.285823 (+ 2727us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":238,"threads_started":1}
W20260929 02:19:31.286209 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:31.278788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:45329 (local address 127.7.170.62:40593)
0929 02:19:31.278970 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:31.278976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:31.279220 (+ 244us) server_negotiation.cc:407] Connection header received
0929 02:19:31.279346 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:31.279349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:31.279409 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:31.279495 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:31.280092 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.280779 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.281917 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.282313 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.282398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.286011 (+ 3613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:45329: BlockingRecv error: recv got EOF from 127.7.170.2:45329 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:31.729095 17109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:31.721416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:41693)
0929 02:19:31.721845 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:31.721863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:31.721970 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:31.722225 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:31.722228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:31.722573 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:31.722722 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.722731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.723633 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.723636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.724717 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.724731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.725491 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.725497 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.725681 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.726234 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:31.726255 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:31.728784 (+ 2529us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":240,"threads_started":1}
W20260929 02:19:31.729177 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:31.721460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41693 (local address 127.7.170.62:40593)
0929 02:19:31.721647 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:31.721653 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:31.721928 (+ 275us) server_negotiation.cc:407] Connection header received
0929 02:19:31.722060 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:31.722064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:31.722122 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:31.722195 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:31.722821 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.723537 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.724852 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.725355 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.725477 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.728994 (+ 3517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:41693: BlockingRecv error: recv got EOF from 127.7.170.3:41693 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:31 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:31.883855 17110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:31.876296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:52855)
0929 02:19:31.876848 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:31.876864 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:31.877006 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:31.877254 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:31.877257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:31.877655 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:31.877807 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.877815 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.878794 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.878798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.879688 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:31.879699 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.880394 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.880400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:31.880566 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.881129 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:31.881146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:31.883627 (+ 2481us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":296,"threads_started":1}
W20260929 02:19:31.883962 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:31.876393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:52855 (local address 127.7.170.62:40593)
0929 02:19:31.876553 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:31.876558 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:31.876942 (+ 384us) server_negotiation.cc:407] Connection header received
0929 02:19:31.877090 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:31.877094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:31.877151 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:31.877226 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:31.877957 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.878664 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.879821 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:31.880262 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:31.880366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:31.883804 (+ 3438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:52855: BlockingRecv error: recv got EOF from 127.7.170.1:52855 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:32.294637 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:32.287049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:43367)
0929 02:19:32.287490 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:32.287506 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:32.287613 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:32.287863 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:32.287865 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:32.288267 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:32.288426 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.288435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.289435 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.289440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.290338 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.290350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.290953 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.290957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.291090 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.291610 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:32.291628 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:32.294412 (+ 2784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":206,"threads_started":1}
W20260929 02:19:32.294803 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:32.287193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:43367 (local address 127.7.170.62:40593)
0929 02:19:32.287401 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:32.287407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:32.287573 (+ 166us) server_negotiation.cc:407] Connection header received
0929 02:19:32.287698 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:32.287702 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:32.287760 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:32.287832 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:32.288533 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.289309 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.290453 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.290846 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.290930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.294598 (+ 3668us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:43367: BlockingRecv error: recv got EOF from 127.7.170.2:43367 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:32.738444 17112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:32.730076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:42759)
0929 02:19:32.730543 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:32.730558 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:32.730679 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:32.730965 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:32.730970 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:32.731378 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:32.731541 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.731550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.732497 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.732501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.733435 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.733447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.734101 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.734107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.734274 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.734821 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:32.734839 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:32.738106 (+ 3267us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":217,"threads_started":1}
W20260929 02:19:32.738631 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:32.730140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:42759 (local address 127.7.170.62:40593)
0929 02:19:32.730319 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:32.730325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:32.730635 (+ 310us) server_negotiation.cc:407] Connection header received
0929 02:19:32.730782 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:32.730787 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:32.730845 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:32.730922 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:32.731668 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.732360 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.733563 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.733970 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.734075 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.738375 (+ 4300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:42759: BlockingRecv error: recv got EOF from 127.7.170.3:42759 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:32 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:32.892460 17113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:32.884705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:51345)
0929 02:19:32.885218 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:32.885234 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:32.885355 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:32.885646 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:32.885650 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:32.886062 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:32.886230 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.886240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.887216 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.887221 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.888110 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:32.888122 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.888778 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.888784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:32.888986 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.889546 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:32.889563 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:32.892162 (+ 2599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":305,"threads_started":1}
W20260929 02:19:32.892566 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:32.884803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:51345 (local address 127.7.170.62:40593)
0929 02:19:32.885051 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:32.885056 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:32.885310 (+ 254us) server_negotiation.cc:407] Connection header received
0929 02:19:32.885462 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:32.885467 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:32.885530 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:32.885607 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:32.886354 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.887086 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.888240 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:32.888640 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:32.888741 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:32.892354 (+ 3613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:51345: BlockingRecv error: recv got EOF from 127.7.170.1:51345 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:33.303565 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:33.295472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:33815)
0929 02:19:33.295967 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:33.295997 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:33.296119 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:33.296406 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:33.296411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:33.296788 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:33.296952 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.296961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.298051 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.298055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.299122 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.299134 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.299919 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.299925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.300093 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.300661 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:33.300679 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:33.303266 (+ 2587us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":269,"threads_started":1}
W20260929 02:19:33.303751 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:33.295565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:33815 (local address 127.7.170.62:40593)
0929 02:19:33.295768 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:33.295774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:33.296074 (+ 300us) server_negotiation.cc:407] Connection header received
0929 02:19:33.296224 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:33.296230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:33.296292 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:33.296366 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:33.297098 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.297930 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.299256 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.299771 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.299890 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.303522 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:33815: BlockingRecv error: recv got EOF from 127.7.170.2:33815 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:33.746886 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:33.739406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:43275)
0929 02:19:33.739889 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:33.739903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:33.740043 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:33.740342 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:33.740347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:33.740764 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:33.740918 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.740956 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.741967 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.741972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.742853 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.742864 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.743565 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.743571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.743725 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.744253 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:33.744270 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:33.746675 (+ 2405us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":278,"threads_started":1}
W20260929 02:19:33.747035 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:33.739500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:43275 (local address 127.7.170.62:40593)
0929 02:19:33.739676 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:33.739683 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:33.739977 (+ 294us) server_negotiation.cc:407] Connection header received
0929 02:19:33.740151 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:33.740156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:33.740215 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:33.740297 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:33.741084 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.741838 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.742984 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.743418 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.743523 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.746830 (+ 3307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:43275: BlockingRecv error: recv got EOF from 127.7.170.3:43275 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:33 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:33.901278 17116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:33.893440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:60355)
0929 02:19:33.894001 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:33.894017 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:33.894124 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:33.894378 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:33.894380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:33.894767 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:33.894923 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.894931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.895832 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.895835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.896786 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:33.896798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.897600 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.897604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:33.897739 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.898255 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:33.898272 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:33.900916 (+ 2644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":244,"threads_started":1}
W20260929 02:19:33.901338 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:33.893509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:60355 (local address 127.7.170.62:40593)
0929 02:19:33.893694 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:33.893699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:33.894078 (+ 379us) server_negotiation.cc:407] Connection header received
0929 02:19:33.894221 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:33.894225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:33.894282 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:33.894350 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:33.895035 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.895732 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.896954 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:33.897473 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:33.897570 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:33.901141 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:60355: BlockingRecv error: recv got EOF from 127.7.170.1:60355 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:34.312395 17117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:34.304391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:34491)
0929 02:19:34.304892 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:34.304907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:34.305048 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:34.305314 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:34.305317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:34.305698 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:34.305870 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.305878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.306826 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.306829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.307725 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.307735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.308320 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.308324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.308431 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.309064 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:34.309087 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:34.312067 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":224,"threads_started":1}
W20260929 02:19:34.312486 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:34.304455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:34491 (local address 127.7.170.62:40593)
0929 02:19:34.304653 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:34.304661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:34.304998 (+ 337us) server_negotiation.cc:407] Connection header received
0929 02:19:34.305151 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:34.305155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:34.305215 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:34.305289 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:34.305976 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.306724 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.307829 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.308212 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.308296 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.312270 (+ 3974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:34491: BlockingRecv error: recv got EOF from 127.7.170.2:34491 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:34.756037 17118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:34.747676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:35177)
0929 02:19:34.748201 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:34.748216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:34.748321 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:34.748608 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:34.748611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:34.749115 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:34.749322 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.749335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.750278 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.750283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.751420 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.751436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.752351 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.752358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.752527 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.753070 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:34.753088 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:34.755682 (+ 2594us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":230,"threads_started":1}
W20260929 02:19:34.756101 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:34.747719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35177 (local address 127.7.170.62:40593)
0929 02:19:34.747867 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:34.747871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:34.748276 (+ 405us) server_negotiation.cc:407] Connection header received
0929 02:19:34.748421 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:34.748425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:34.748508 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:34.748574 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:34.749436 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.750139 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.751558 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.752179 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.752275 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.755883 (+ 3608us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:35177: BlockingRecv error: recv got EOF from 127.7.170.3:35177 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:34 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:34.915829 17119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:34.902268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:33709)
0929 02:19:34.902755 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:34.902774 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:34.902908 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:34.903235 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:34.903239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:34.903583 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:34.903726 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.903734 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.904631 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.904635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.905579 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:34.905591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.906254 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.906258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:34.906363 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.912574 (+ 6211us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:34.912593 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:34.915520 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":385,"threads_started":1}
W20260929 02:19:34.916066 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:34.902454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:33709 (local address 127.7.170.62:40593)
0929 02:19:34.902646 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:34.902652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:34.902857 (+ 205us) server_negotiation.cc:407] Connection header received
0929 02:19:34.903067 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:34.903071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:34.903129 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:34.903200 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:34.903828 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.904521 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.905683 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:34.906124 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:34.906209 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:34.915712 (+ 9503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:33709: BlockingRecv error: recv got EOF from 127.7.170.1:33709 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:35.321218 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:35.313325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:42193)
0929 02:19:35.313795 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:35.313811 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:35.313937 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:35.314233 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:35.314238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:35.314606 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:35.314748 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.314757 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.315857 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.315860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.316760 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.316772 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.317511 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.317517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.317677 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.318213 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:35.318230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:35.320896 (+ 2666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":216,"threads_started":1}
W20260929 02:19:35.321305 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:35.313383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42193 (local address 127.7.170.62:40593)
0929 02:19:35.313576 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:35.313582 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:35.313893 (+ 311us) server_negotiation.cc:407] Connection header received
0929 02:19:35.314045 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:35.314051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:35.314113 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:35.314189 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:35.314889 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.315724 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.316894 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.317346 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.317453 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.321120 (+ 3667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:42193: BlockingRecv error: recv got EOF from 127.7.170.2:42193 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:35.764369 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:35.756981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:48809)
0929 02:19:35.757443 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:35.757458 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:35.757587 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:35.757894 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:35.757897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:35.758226 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:35.758367 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.758375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.759271 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.759275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.760182 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.760192 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.760834 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.760839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.760977 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.761560 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:35.761582 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:35.764156 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":211,"threads_started":1}
W20260929 02:19:35.764540 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:35.757059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:48809 (local address 127.7.170.62:40593)
0929 02:19:35.757275 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:35.757281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:35.757539 (+ 258us) server_negotiation.cc:407] Connection header received
0929 02:19:35.757689 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:35.757694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:35.757785 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:35.757884 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:35.758469 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.759164 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.760287 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.760692 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.760776 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.764325 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:48809: BlockingRecv error: recv got EOF from 127.7.170.3:48809 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:35 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:35.924249 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:35.916636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:52519)
0929 02:19:35.917140 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:35.917159 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:35.917296 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:35.917615 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:35.917618 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:35.918057 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:35.918252 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.918266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.919310 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.919314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.920241 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:35.920253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.920858 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.920862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:35.921027 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.921502 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:35.921519 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:35.924013 (+ 2494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":265,"threads_started":1}
W20260929 02:19:35.924397 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:35.916702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:52519 (local address 127.7.170.62:40593)
0929 02:19:35.916861 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:35.916867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:35.917230 (+ 363us) server_negotiation.cc:407] Connection header received
0929 02:19:35.917423 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:35.917427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:35.917491 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:35.917592 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:35.918386 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.919196 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.920353 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:35.920750 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:35.920862 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:35.924193 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:52519: BlockingRecv error: recv got EOF from 127.7.170.1:52519 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:36.330650 17123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:36.322068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:40443)
0929 02:19:36.322558 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:36.322574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:36.322700 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:36.323013 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:36.323018 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:36.323423 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:36.323626 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.323637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.324621 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.324625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.325793 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.325826 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.326657 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.326663 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.326799 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.327357 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:36.327375 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:36.330329 (+ 2954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":268,"threads_started":1}
W20260929 02:19:36.330775 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:36.322149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:40443 (local address 127.7.170.62:40593)
0929 02:19:36.322325 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:36.322332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:36.322655 (+ 323us) server_negotiation.cc:407] Connection header received
0929 02:19:36.322804 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:36.322809 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:36.322868 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:36.322945 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:36.323766 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.324491 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.325937 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.326476 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.326593 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.330552 (+ 3959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:40443: BlockingRecv error: recv got EOF from 127.7.170.2:40443 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:36.773005 17124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:36.765164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:48683)
0929 02:19:36.765640 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:36.765656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:36.765781 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:36.766134 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:36.766139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:36.766532 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:36.766694 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.766703 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.767670 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.767676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.768610 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.768622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.769468 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.769474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.769623 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.770271 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:36.770289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:36.772718 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":256,"threads_started":1}
W20260929 02:19:36.773114 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:36.765232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:48683 (local address 127.7.170.62:40593)
0929 02:19:36.765397 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:36.765403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:36.765730 (+ 327us) server_negotiation.cc:407] Connection header received
0929 02:19:36.765904 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:36.765909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:36.765972 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:36.766087 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:36.766824 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.767550 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.768738 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.769299 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.769414 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.772888 (+ 3474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:48683: BlockingRecv error: recv got EOF from 127.7.170.3:48683 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:36 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:36.932386 17125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:36.925097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:58377)
0929 02:19:36.925527 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:36.925543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:36.925651 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:36.925928 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:36.925930 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:36.926273 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:36.926410 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.926418 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.927314 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.927318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.928216 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:36.928226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.928838 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.928842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:36.928967 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.929482 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:36.929498 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:36.932166 (+ 2668us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":236,"threads_started":1}
W20260929 02:19:36.932574 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:36.925151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:58377 (local address 127.7.170.62:40593)
0929 02:19:36.925334 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:36.925341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:36.925629 (+ 288us) server_negotiation.cc:407] Connection header received
0929 02:19:36.925737 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:36.925741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:36.925826 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:36.925901 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:36.926514 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.927210 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.928315 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:36.928734 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:36.928815 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:36.932347 (+ 3532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:58377: BlockingRecv error: recv got EOF from 127.7.170.1:58377 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:37.339399 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:37.331658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:54913)
0929 02:19:37.332134 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:37.332153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:37.332274 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:37.332552 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:37.332557 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:37.333010 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:37.333184 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.333195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.334196 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.334202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.335340 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.335352 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.335987 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.335992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.336102 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.336616 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:37.336632 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:37.339186 (+ 2554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":264,"threads_started":1}
W20260929 02:19:37.339596 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:37.331670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54913 (local address 127.7.170.62:40593)
0929 02:19:37.331861 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:37.331867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:37.332221 (+ 354us) server_negotiation.cc:407] Connection header received
0929 02:19:37.332362 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:37.332366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:37.332423 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:37.332503 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:37.333326 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.334073 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.335470 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.335875 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.335964 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.339367 (+ 3403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:54913: BlockingRecv error: recv got EOF from 127.7.170.2:54913 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:37.781605 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:37.773838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:52533)
0929 02:19:37.774283 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:37.774297 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:37.774415 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:37.774710 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:37.774715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:37.775080 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:37.775221 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.775230 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.776206 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.776211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.777152 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.777163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.777829 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.777835 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.777982 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.778519 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:37.778535 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:37.781325 (+ 2790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
W20260929 02:19:37.781773 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:37.774004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:52533 (local address 127.7.170.62:40593)
0929 02:19:37.774196 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:37.774202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:37.774370 (+ 168us) server_negotiation.cc:407] Connection header received
0929 02:19:37.774520 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:37.774526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:37.774592 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:37.774668 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:37.775359 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.776078 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.777286 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.777685 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.777790 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.781544 (+ 3754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:52533: BlockingRecv error: recv got EOF from 127.7.170.3:52533 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:37 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:37.940655 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:37.933197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:49469)
0929 02:19:37.933672 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:37.933687 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:37.933795 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:37.934066 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:37.934069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:37.934435 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:37.934607 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.934616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.935531 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.935534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.936400 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:37.936410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.937109 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.937113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:37.937254 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.937771 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:37.937788 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:37.940407 (+ 2619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":266,"threads_started":1}
W20260929 02:19:37.940814 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:37.933252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:49469 (local address 127.7.170.62:40593)
0929 02:19:37.933421 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:37.933425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:37.933750 (+ 325us) server_negotiation.cc:407] Connection header received
0929 02:19:37.933876 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:37.933880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:37.933949 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:37.934028 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:37.934729 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.935428 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.936511 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:37.936953 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:37.937035 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:37.940603 (+ 3568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:49469: BlockingRecv error: recv got EOF from 127.7.170.1:49469 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:38.347988 17129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:38.340191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:46747)
0929 02:19:38.340626 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:38.340643 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:38.340750 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:38.341064 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:38.341069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:38.341474 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:38.341637 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.341646 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.342608 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.342613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.343501 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.343512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.344201 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.344207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.344380 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.344903 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:38.344919 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:38.347706 (+ 2787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":233,"threads_started":1}
W20260929 02:19:38.348157 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:38.340277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:46747 (local address 127.7.170.62:40593)
0929 02:19:38.340480 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:38.340486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:38.340712 (+ 226us) server_negotiation.cc:407] Connection header received
0929 02:19:38.340833 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:38.340836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:38.340901 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:38.341017 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:38.341768 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.342468 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.343632 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.344061 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.344167 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.347907 (+ 3740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:46747: BlockingRecv error: recv got EOF from 127.7.170.2:46747 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:38.790182 17130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:38.782334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:56099)
0929 02:19:38.782814 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:38.782830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:38.782936 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:38.783180 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:38.783183 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:38.783565 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:38.783738 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.783747 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.784707 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.784711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.785714 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.785729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.786498 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.786504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.786655 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.787180 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:38.787199 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:38.789860 (+ 2661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":265,"threads_started":1}
W20260929 02:19:38.790290 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:38.782406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:56099 (local address 127.7.170.62:40593)
0929 02:19:38.782560 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:38.782564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:38.782889 (+ 325us) server_negotiation.cc:407] Connection header received
0929 02:19:38.783020 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:38.783024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:38.783080 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:38.783153 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:38.783855 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.784588 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.785858 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.786343 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.786449 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.790083 (+ 3634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:56099: BlockingRecv error: recv got EOF from 127.7.170.3:56099 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:38 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:38.949044 17131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:38.941518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:49547)
0929 02:19:38.941949 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:38.941965 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:38.942091 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:38.942361 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:38.942364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:38.942717 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:38.942868 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.942876 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.943770 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.943773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.944665 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:38.944676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.945338 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.945343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:38.945467 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.946000 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:38.946016 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:38.948708 (+ 2692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":217,"threads_started":1}
W20260929 02:19:38.949151 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:38.941593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:49547 (local address 127.7.170.62:40593)
0929 02:19:38.941795 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:38.941801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:38.942051 (+ 250us) server_negotiation.cc:407] Connection header received
0929 02:19:38.942190 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:38.942193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:38.942254 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:38.942327 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:38.942970 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.943672 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.944766 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:38.945221 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:38.945302 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:38.948911 (+ 3609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:49547: BlockingRecv error: recv got EOF from 127.7.170.1:49547 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:39.356194 17132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:39.348888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:54737)
0929 02:19:39.349467 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:39.349490 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:39.349596 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:39.349836 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:39.349839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:39.350195 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:39.350348 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.350356 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.351225 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.351228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.352127 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.352139 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.352762 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.352766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.352885 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.353467 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:39.353492 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:39.355966 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":340,"threads_started":1}
W20260929 02:19:39.356312 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:39.348914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54737 (local address 127.7.170.62:40593)
0929 02:19:39.349127 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:39.349134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:39.349563 (+ 429us) server_negotiation.cc:407] Connection header received
0929 02:19:39.349680 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:39.349683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:39.349743 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:39.349815 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:39.350445 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.351126 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.352231 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.352665 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.352749 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.356143 (+ 3394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:54737: BlockingRecv error: recv got EOF from 127.7.170.2:54737 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:39.799067 17133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:39.791143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:45827)
0929 02:19:39.791603 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:39.791618 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:39.791741 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:39.792042 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:39.792047 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:39.792392 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:39.792549 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.792559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.793588 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.793592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.794501 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.794513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.795148 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.795153 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.795313 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.795812 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:39.795828 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:39.798795 (+ 2967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
W20260929 02:19:39.799161 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:39.791169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:45827 (local address 127.7.170.62:40593)
0929 02:19:39.791324 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:39.791330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:39.791696 (+ 366us) server_negotiation.cc:407] Connection header received
0929 02:19:39.791848 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:39.791852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:39.791910 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:39.791983 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:39.792684 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.793448 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.794627 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.795022 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.795122 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.798969 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:45827: BlockingRecv error: recv got EOF from 127.7.170.3:45827 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:39 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:39.966606 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:39.949881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:47107)
0929 02:19:39.950387 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:39.950402 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:39.950521 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:39.950768 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:39.950771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:39.951156 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:39.951307 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.951316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.952202 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.952205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.953161 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:39.953172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.953799 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.953803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:39.953910 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.963585 (+ 9675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:39.963603 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:39.966294 (+ 2691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
W20260929 02:19:39.966727 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:39.949944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:47107 (local address 127.7.170.62:40593)
0929 02:19:39.950142 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:39.950148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:39.950470 (+ 322us) server_negotiation.cc:407] Connection header received
0929 02:19:39.950607 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:39.950610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:39.950681 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:39.950754 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:39.951406 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.952107 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.953258 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:39.953675 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:39.953757 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:39.966505 (+ 12748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:47107: BlockingRecv error: recv got EOF from 127.7.170.1:47107 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:40.364923 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:40.357017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:41313)
0929 02:19:40.357497 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:40.357513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:40.357636 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:40.357920 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:40.357924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:40.358293 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:40.358446 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.358456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.359446 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.359452 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.360362 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.360374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.361099 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.361105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.361253 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.361797 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:40.361812 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:40.364545 (+ 2733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":234,"threads_started":1}
W20260929 02:19:40.364946 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:40.357130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:41313 (local address 127.7.170.62:40593)
0929 02:19:40.357310 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:40.357316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:40.357590 (+ 274us) server_negotiation.cc:407] Connection header received
0929 02:19:40.357739 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:40.357744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:40.357801 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:40.357876 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:40.358590 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.359319 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.360495 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.360909 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.361053 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.364709 (+ 3656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:41313: BlockingRecv error: recv got EOF from 127.7.170.2:41313 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:40.808209 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:40.799843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:52857)
0929 02:19:40.800326 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:40.800341 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:40.800447 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:40.800696 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:40.800699 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:40.801096 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:40.801253 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.801262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.802190 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.802195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.803125 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.803136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.803969 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.803982 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.804122 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.805028 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:40.805051 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:40.807883 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":252,"threads_started":1}
W20260929 02:19:40.808293 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:40.799896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:52857 (local address 127.7.170.62:40593)
0929 02:19:40.800037 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:40.800042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:40.800401 (+ 359us) server_negotiation.cc:407] Connection header received
0929 02:19:40.800532 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:40.800536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:40.800593 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:40.800667 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:40.801354 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.802068 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.803229 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.803632 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.803742 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.808082 (+ 4340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:52857: BlockingRecv error: recv got EOF from 127.7.170.3:52857 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:40 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:40.974797 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:40.967422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:45651)
0929 02:19:40.967866 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:40.967882 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:40.968021 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:40.968337 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:40.968341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:40.968659 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:40.968824 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.968832 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.969838 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.969843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.970744 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:40.970758 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.971394 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.971398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:40.971520 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.972022 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:40.972038 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:40.974557 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":221,"threads_started":1}
W20260929 02:19:40.974911 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:40.967522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:45651 (local address 127.7.170.62:40593)
0929 02:19:40.967811 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:40.967817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:40.967950 (+ 133us) server_negotiation.cc:407] Connection header received
0929 02:19:40.968140 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:40.968145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:40.968222 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:40.968294 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:40.968953 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.969705 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.970888 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:40.971286 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:40.971374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:40.974728 (+ 3354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:45651: BlockingRecv error: recv got EOF from 127.7.170.1:45651 (error 108)
Metrics: {"server-negotiator.queue_time_us":168}
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:41.373724 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:41.365832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:58773)
0929 02:19:41.366285 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:41.366300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:41.366413 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:41.366668 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:41.366670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:41.367017 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:41.367188 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.367200 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.368118 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.368122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.369063 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.369075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.369784 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.369788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.369903 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.370476 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:41.370495 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:41.373473 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":243,"threads_started":1}
W20260929 02:19:41.373852 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:41.365901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:58773 (local address 127.7.170.62:40593)
0929 02:19:41.366031 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:41.366036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:41.366364 (+ 328us) server_negotiation.cc:407] Connection header received
0929 02:19:41.366511 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:41.366514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:41.366572 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:41.366639 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:41.367310 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.368002 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.369215 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.369641 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.369783 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.373646 (+ 3863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:58773: BlockingRecv error: recv got EOF from 127.7.170.2:58773 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.3@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:41.817129 17139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:41.809033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.3:38863)
0929 02:19:41.809516 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:41.809532 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:41.809651 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:41.809963 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:41.809969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:41.810326 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:41.810626 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.810642 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.811809 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.811815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.813041 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.813053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.813694 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.813698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.813805 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.814374 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:41.814396 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:41.816839 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":236,"threads_started":1}
W20260929 02:19:41.817256 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:41.809123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:38863 (local address 127.7.170.62:40593)
0929 02:19:41.809315 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:41.809321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:41.809605 (+ 284us) server_negotiation.cc:407] Connection header received
0929 02:19:41.809774 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:41.809778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:41.809841 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:41.809916 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:41.810797 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.811658 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.813177 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.813582 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.813668 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.817077 (+ 3409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.3:38863: BlockingRecv error: recv got EOF from 127.7.170.3:38863 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:41 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.1@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:41.983465 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:41.975813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.1:36219)
0929 02:19:41.976280 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:41.976295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:41.976410 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:41.976720 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:41.976725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:41.977139 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:41.977323 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.977332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.978314 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.978319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.979206 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:41.979217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.979915 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.979920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:41.980093 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.980591 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:41.980607 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:41.983129 (+ 2522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":270,"threads_started":1}
W20260929 02:19:41.983566 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:41.975907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:36219 (local address 127.7.170.62:40593)
0929 02:19:41.976078 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:41.976084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:41.976369 (+ 285us) server_negotiation.cc:407] Connection header received
0929 02:19:41.976525 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:41.976530 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:41.976602 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:41.976677 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:41.977468 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.978186 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.979334 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:41.979786 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:41.979888 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:41.983335 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.1:36219: BlockingRecv error: recv got EOF from 127.7.170.1:36219 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 29 02:19:42 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
Sep 29 02:19:42 dist-test-slave-8cs2 krb5kdc[16094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.170.2@KRBTEST.COM for oryx/127.7.170.62@KRBTEST.COM, Server not found in Kerberos database
I20260929 02:19:42.382570 17141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:42.374649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:40593 (local address 127.7.170.2:36649)
0929 02:19:42.375141 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:42.375156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:42.375275 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:42.375581 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:42.375586 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:42.375972 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:42.376162 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:42.376170 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.377186 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.377190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:42.378102 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:42.378113 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.378873 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.378878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:42.379037 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:42.379631 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:42.379648 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:42.382228 (+ 2580us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.170.62:40593: Server oryx/127.7.170.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":278,"threads_started":1}
W20260929 02:19:42.382653 16832 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:42.374736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:36649 (local address 127.7.170.62:40593)
0929 02:19:42.374902 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:42.374909 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:42.375233 (+ 324us) server_negotiation.cc:407] Connection header received
0929 02:19:42.375381 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:42.375385 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:42.375442 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:42.375537 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:42.376294 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.377050 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.378229 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.378743 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.378848 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:42.382423 (+ 3575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.170.2:36649: BlockingRecv error: recv got EOF from 127.7.170.2:36649 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:19:42.669538 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16683
I20260929 02:19:42.680624 17143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:42.670454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.7.170.1:34071)
0929 02:19:42.671042 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:42.671050 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:42.671077 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:19:42.671135 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:42.671142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:42.671349 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:42.671507 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:42.672765 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.673817 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.675013 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.675566 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.675659 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:42.677856 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:42.677864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:42.677878 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:42.677912 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:42.680166 (+ 2254us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:19:42.680411 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45882: Request ticket server kudu/127.7.170.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":290,"threads_started":1}
W20260929 02:19:42.680717 17143 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45882: Request ticket server kudu/127.7.170.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:19:42.680752 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16683: Not authorized: Client connection negotiation failed: client connection to 127.7.170.1:34071: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:19:42.680985 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16683
I20260929 02:19:42.694684 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16834
I20260929 02:19:42.705992 17144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:42.695630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43338 (local address 127.7.170.2:35181)
0929 02:19:42.696162 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:42.696169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:42.696199 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:19:42.696256 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:42.696262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:42.696501 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:42.696683 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:42.698029 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.699140 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.700276 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.700784 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.700874 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:42.703154 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:42.703162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:42.703176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:42.703209 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:42.705507 (+ 2298us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:19:42.705731 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43338: Request ticket server kudu/127.7.170.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":217,"threads_started":1}
W20260929 02:19:42.706087 17144 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43338: Request ticket server kudu/127.7.170.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:19:42.706089 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16834: Not authorized: Client connection negotiation failed: client connection to 127.7.170.2:35181: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:19:42.706235 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16834
I20260929 02:19:42.718729 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16965
I20260929 02:19:42.729096 17145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:42.719497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43600 (local address 127.7.170.3:45623)
0929 02:19:42.720064 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:42.720071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:42.720095 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:19:42.720152 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:42.720158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:42.720360 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:42.720518 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:42.721634 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.722659 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.723699 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:42.724209 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:42.724297 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:42.726466 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:42.726476 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:42.726502 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:42.726549 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:42.728602 (+ 2053us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0929 02:19:42.728833 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43600: Request ticket server kudu/127.7.170.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":307,"threads_started":1}
W20260929 02:19:42.729190 17145 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43600: Request ticket server kudu/127.7.170.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260929 02:19:42.729169 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16965: Not authorized: Client connection negotiation failed: client connection to 127.7.170.3:45623: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.7.170.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260929 02:19:42.729338 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16965
I20260929 02:19:42.745354 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 16614
I20260929 02:19:42.746490 16676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:42.856539 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 16614
2026-09-29T02:19:42Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (20667 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-29T02:19:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:42Z Disabled control of system clock
I20260929 02:19:42.924229 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42449
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42449
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260929 02:19:43.230074 17150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:43.230340 17150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:43.230453 17150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:43.237561 17150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:43.237659 17150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:43.237726 17150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:43.237800 17150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:43.237854 17150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:43.246767 17150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42449
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:43.247991 17150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:43.249734 17150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:43.260784 17158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:43.260845 17155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:43.260825 17156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:43.261091 17150 server_base.cc:1034] running on GCE node
I20260929 02:19:43.262060 17150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:43.264238 17150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:43.265436 17150 hybrid_clock.cc:648] HybridClock initialized: now 1790648383265367 us; error 61 us; skew 500 ppm
I20260929 02:19:43.265749 17150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:43.267781 17150 webserver.cc:460] Webserver started at http://127.7.170.62:36573/ using document root <none> and password file <none>
I20260929 02:19:43.268414 17150 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:43.268507 17150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:43.268797 17150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:43.271482 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "878a597d0f934b5bb6e7c1ab543407b6"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "2dd563ec5d1fa614326cf5c3c83a3466"
server_key_iv: "3b6bfa9df4e600e430f276b553eaec59"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.272207 17150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "878a597d0f934b5bb6e7c1ab543407b6"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "2dd563ec5d1fa614326cf5c3c83a3466"
server_key_iv: "3b6bfa9df4e600e430f276b553eaec59"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.277146 17150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:19:43.280187 17164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:43.281426 17150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:19:43.281572 17150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "878a597d0f934b5bb6e7c1ab543407b6"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "2dd563ec5d1fa614326cf5c3c83a3466"
server_key_iv: "3b6bfa9df4e600e430f276b553eaec59"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.281746 17150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:43.292105 17150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:43.292789 17150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:43.293099 17150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:43.308918 17150 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42449
I20260929 02:19:43.308954 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42449 every 8 connection(s)
I20260929 02:19:43.310207 17150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:43.312783 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17150
I20260929 02:19:43.312997 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:43.313812 7848 external_mini_cluster.cc:1428] Setting key 07ff49c677358c3e1846dfe9e2101e4c
I20260929 02:19:43.314268 17216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:43.326735 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6: Bootstrap starting.
I20260929 02:19:43.326846 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:43.315773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46844 (local address 127.7.170.62:42449)
0929 02:19:43.316692 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:43.316712 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:43.316752 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:19:43.318531 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:43.318552 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:43.318988 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:43.319295 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:43.319858 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.321160 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.322832 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.323448 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.323582 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:43.324733 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:43.324739 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:43.324751 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:43.325290 (+ 539us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:43.325361 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:43.325379 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:43.325457 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:43.325660 (+ 203us) server_negotiation.cc:299] Negotiation successful
0929 02:19:43.325944 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":324,"threads_started":1}
I20260929 02:19:43.329603 17216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:43.330492 17216 log.cc:824] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:43.332857 17216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6: No bootstrap required, opened a new log
I20260929 02:19:43.339370 17216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } }
I20260929 02:19:43.339663 17216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:43.339747 17216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878a597d0f934b5bb6e7c1ab543407b6, State: Initialized, Role: FOLLOWER
I20260929 02:19:43.340232 17216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } }
I20260929 02:19:43.340417 17216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:43.340533 17216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:43.340677 17216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:43.342631 17216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } }
I20260929 02:19:43.343077 17216 leader_election.cc:302] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 878a597d0f934b5bb6e7c1ab543407b6; no voters:
I20260929 02:19:43.343539 17216 leader_election.cc:288] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:43.343658 17221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:43.344292 17221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [term 1 LEADER]: Becoming Leader. State: Replica: 878a597d0f934b5bb6e7c1ab543407b6, State: Running, Role: LEADER
I20260929 02:19:43.344761 17221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } }
I20260929 02:19:43.345324 17216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:43.347625 17222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "878a597d0f934b5bb6e7c1ab543407b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } } }
I20260929 02:19:43.347653 17223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 878a597d0f934b5bb6e7c1ab543407b6. Latest consensus state: current_term: 1 leader_uuid: "878a597d0f934b5bb6e7c1ab543407b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "878a597d0f934b5bb6e7c1ab543407b6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42449 } } }
I20260929 02:19:43.348003 17222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:43.348011 17223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:43.354156 17229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:43.358870 17229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:43.366369 17229 catalog_manager.cc:1347] Generated new cluster ID: 84bece25bcb1460a9331e571af1a1d0e
I20260929 02:19:43.366466 17229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:43.386636 17229 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:43.387841 17229 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:43.397270 17229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 878a597d0f934b5bb6e7c1ab543407b6: Generated new TSK 0
I20260929 02:19:43.398056 17229 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:19:43.410183 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--builtin_ntp_servers=127.7.170.20:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260929 02:19:43.720469 17240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:43.720727 17240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:43.720839 17240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:43.728034 17240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:43.728121 17240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:43.728253 17240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:43.737047 17240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:43.738237 17240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:43.739952 17240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:43.751955 17245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:43.751994 17248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:43.751920 17246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:43.752668 17240 server_base.cc:1034] running on GCE node
I20260929 02:19:43.753243 17240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:43.754076 17240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:43.755272 17240 hybrid_clock.cc:648] HybridClock initialized: now 1790648383755222 us; error 42 us; skew 500 ppm
I20260929 02:19:43.755555 17240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:43.757702 17240 webserver.cc:460] Webserver started at http://127.7.170.1:39775/ using document root <none> and password file <none>
I20260929 02:19:43.758325 17240 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:43.758428 17240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:43.758718 17240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:43.761298 17240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "cad022be6a2e4afb965eb6cd716d7d80"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "e754120e6afa746257d32e8217bfd51c"
server_key_iv: "865988ffd1d6f6493dc2cbf490803be0"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.762008 17240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "cad022be6a2e4afb965eb6cd716d7d80"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "e754120e6afa746257d32e8217bfd51c"
server_key_iv: "865988ffd1d6f6493dc2cbf490803be0"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.766569 17240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:43.769615 17254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:43.770776 17240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:19:43.770920 17240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "cad022be6a2e4afb965eb6cd716d7d80"
format_stamp: "Formatted at 2026-09-29 02:19:43 on dist-test-slave-8cs2"
server_key: "e754120e6afa746257d32e8217bfd51c"
server_key_iv: "865988ffd1d6f6493dc2cbf490803be0"
server_key_version: "encryptionkey@0"
I20260929 02:19:43.771106 17240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:43.783142 17240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:43.783850 17240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:43.784071 17240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:43.784951 17240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:43.786245 17240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:43.786334 17240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:43.786437 17240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:43.786513 17240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:43.814221 17240 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:34275
I20260929 02:19:43.814265 17366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:34275 every 8 connection(s)
I20260929 02:19:43.815431 17240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:43.820287 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17240
I20260929 02:19:43.820441 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:43.821125 7848 external_mini_cluster.cc:1428] Setting key cd7e382440d05e487df904a83d95ff36
I20260929 02:19:43.823699 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--builtin_ntp_servers=127.7.170.20:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:19:43.827271 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:43.817783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:40801 (local address 127.7.170.62:42449)
0929 02:19:43.818008 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:43.818015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:43.819247 (+ 1232us) server_negotiation.cc:407] Connection header received
0929 02:19:43.820949 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:43.820957 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:43.821050 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:43.821160 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:43.822248 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.823077 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.824616 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.825217 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.825330 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:43.826580 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:43.826586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:43.826590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:43.826609 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:43.826643 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:43.826656 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:43.826740 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:43.826928 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 02:19:43.827016 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:19:43.828260 17369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:43.818262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42449 (local address 127.7.170.1:40801)
0929 02:19:43.819102 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:43.819141 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:43.820675 (+ 1534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:43.821360 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:43.821368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:43.821450 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:43.822107 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:43.822131 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.823217 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.823221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:43.824469 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:43.824482 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:43.825384 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:43.825389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:43.825586 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:43.826260 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:43.826263 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:43.826307 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:43.826388 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:43.826740 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:43.826747 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:43.826753 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:19:43.826904 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:19:43.827226 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":254,"threads_started":1}
I20260929 02:19:43.829602 17367 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42449
I20260929 02:19:43.829901 17367 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:43.830690 17367 heartbeater.cc:511] Master 127.7.170.62:42449 requested a full tablet report, sending...
I20260929 02:19:43.832957 17181 ts_manager.cc:194] Registered new tserver with Master: cad022be6a2e4afb965eb6cd716d7d80 (127.7.170.1:34275)
I20260929 02:19:43.834368 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.1:40801
W20260929 02:19:44.130541 17370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:44.130805 17370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:44.130908 17370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:44.137811 17370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:44.137904 17370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:44.138046 17370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:44.146688 17370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:44.147830 17370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:44.149533 17370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:44.161226 17376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:44.161326 17375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:44.161269 17378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:44.161772 17370 server_base.cc:1034] running on GCE node
I20260929 02:19:44.162377 17370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:44.163183 17370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:44.164390 17370 hybrid_clock.cc:648] HybridClock initialized: now 1790648384164334 us; error 47 us; skew 500 ppm
I20260929 02:19:44.164666 17370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:44.166676 17370 webserver.cc:460] Webserver started at http://127.7.170.2:44231/ using document root <none> and password file <none>
I20260929 02:19:44.167294 17370 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:44.167387 17370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:44.167667 17370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:44.170260 17370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "071e0aca47b44861a85b0bf646feda63"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "ac812df5291c3d134dc092d3cb0d4e8e"
server_key_iv: "e975662941d960088e4bd806ab651aba"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.170960 17370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "071e0aca47b44861a85b0bf646feda63"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "ac812df5291c3d134dc092d3cb0d4e8e"
server_key_iv: "e975662941d960088e4bd806ab651aba"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.175530 17370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260929 02:19:44.178586 17384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.179744 17370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:19:44.179876 17370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "071e0aca47b44861a85b0bf646feda63"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "ac812df5291c3d134dc092d3cb0d4e8e"
server_key_iv: "e975662941d960088e4bd806ab651aba"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.180060 17370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:44.195271 17370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:44.195995 17370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:44.196216 17370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:44.197072 17370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:44.198338 17370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:44.198426 17370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.198527 17370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:44.198593 17370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.226322 17370 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:35983
I20260929 02:19:44.226366 17496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:35983 every 8 connection(s)
I20260929 02:19:44.227509 17370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:44.235308 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17370
I20260929 02:19:44.235437 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:44.235831 7848 external_mini_cluster.cc:1428] Setting key 86ab07df0336173967eab8f9e12764a4
I20260929 02:19:44.238209 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--builtin_ntp_servers=127.7.170.20:38939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260929 02:19:44.238962 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.229791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:39361 (local address 127.7.170.62:42449)
0929 02:19:44.229945 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.229949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.231244 (+ 1295us) server_negotiation.cc:407] Connection header received
0929 02:19:44.232990 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.232996 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.233069 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.233153 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.234164 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.234850 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.236397 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.236894 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.237043 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.238283 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:44.238289 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:44.238293 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:44.238308 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:44.238344 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:44.238358 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:44.238436 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:44.238672 (+ 236us) server_negotiation.cc:299] Negotiation successful
0929 02:19:44.238744 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:19:44.240275 17499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.230283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42449 (local address 127.7.170.2:39361)
0929 02:19:44.231107 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:44.231140 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:44.232721 (+ 1581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:44.233375 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:44.233383 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:44.233467 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:44.234011 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:44.234028 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.234987 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.234991 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:44.236268 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:44.236280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.237133 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.237140 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:44.237401 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.238002 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:44.238005 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:44.238042 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:44.238111 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:44.238464 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:44.238475 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:44.238491 (+ 16us) client_negotiation.cc:769] Sending connection context
0929 02:19:44.238662 (+ 171us) client_negotiation.cc:240] Negotiation successful
0929 02:19:44.239023 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":246,"threads_started":1}
I20260929 02:19:44.241518 17497 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42449
I20260929 02:19:44.241804 17497 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:44.242623 17497 heartbeater.cc:511] Master 127.7.170.62:42449 requested a full tablet report, sending...
I20260929 02:19:44.243956 17181 ts_manager.cc:194] Registered new tserver with Master: 071e0aca47b44861a85b0bf646feda63 (127.7.170.2:35983)
I20260929 02:19:44.244956 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.2:39361
W20260929 02:19:44.542876 17500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:44.543123 17500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:44.543234 17500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:44.550318 17500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:44.550403 17500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:44.550542 17500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:44.558991 17500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42449
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:44.560118 17500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:44.561757 17500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:44.573966 17506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:44.574087 17505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:44.574066 17508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:44.574904 17500 server_base.cc:1034] running on GCE node
I20260929 02:19:44.575506 17500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:44.576339 17500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:44.577526 17500 hybrid_clock.cc:648] HybridClock initialized: now 1790648384577473 us; error 47 us; skew 500 ppm
I20260929 02:19:44.577795 17500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:44.579811 17500 webserver.cc:460] Webserver started at http://127.7.170.3:38037/ using document root <none> and password file <none>
I20260929 02:19:44.580426 17500 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:44.580528 17500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:44.580825 17500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:44.583491 17500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "e78d6bc26b124501a49e91c8c92e132c"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "7f4994fdb5fcca9eaa1b70aad140b9aa"
server_key_iv: "a732e8364df909e410a73ead9f49c73a"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.584234 17500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "e78d6bc26b124501a49e91c8c92e132c"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "7f4994fdb5fcca9eaa1b70aad140b9aa"
server_key_iv: "a732e8364df909e410a73ead9f49c73a"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.589025 17500 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:19:44.591995 17514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.593207 17500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:19:44.593345 17500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e78d6bc26b124501a49e91c8c92e132c"
format_stamp: "Formatted at 2026-09-29 02:19:44 on dist-test-slave-8cs2"
server_key: "7f4994fdb5fcca9eaa1b70aad140b9aa"
server_key_iv: "a732e8364df909e410a73ead9f49c73a"
server_key_version: "encryptionkey@0"
I20260929 02:19:44.593531 17500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:44.624552 17500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:44.625353 17500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:44.625608 17500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:44.626500 17500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:44.627873 17500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:44.627964 17500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.628067 17500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:44.628134 17500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:44.656409 17500 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:46835
I20260929 02:19:44.656469 17626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:46835 every 8 connection(s)
I20260929 02:19:44.657641 17500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:44.659233 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17500
I20260929 02:19:44.659386 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:44.659861 7848 external_mini_cluster.cc:1428] Setting key 5563bed79fd6e0b480315a80fb6a9380
I20260929 02:19:44.669631 17219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.660142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41295 (local address 127.7.170.62:42449)
0929 02:19:44.660334 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.660340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.661970 (+ 1630us) server_negotiation.cc:407] Connection header received
0929 02:19:44.663826 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.663831 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.663910 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.663999 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.665028 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.665720 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.667214 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.667602 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.667706 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.668865 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:44.668872 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:44.668875 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:44.668892 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:44.668921 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:44.668957 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:44.669041 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:44.669226 (+ 185us) server_negotiation.cc:299] Negotiation successful
0929 02:19:44.669313 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:19:44.670554 17629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.660793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42449 (local address 127.7.170.3:41295)
0929 02:19:44.661823 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:44.661873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:44.663588 (+ 1715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:44.664224 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:44.664232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:44.664315 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:44.664856 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:44.664872 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.665877 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.665882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:44.667083 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:44.667095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.667802 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.667809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:44.668020 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.668587 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:44.668589 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:44.668625 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:44.668695 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:44.669039 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:44.669049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:44.669058 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:19:44.669206 (+ 148us) client_negotiation.cc:240] Negotiation successful
0929 02:19:44.669580 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":304,"threads_started":1}
I20260929 02:19:44.671762 17627 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42449
I20260929 02:19:44.672020 17627 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:44.672708 17627 heartbeater.cc:511] Master 127.7.170.62:42449 requested a full tablet report, sending...
I20260929 02:19:44.674146 17181 ts_manager.cc:194] Registered new tserver with Master: e78d6bc26b124501a49e91c8c92e132c (127.7.170.3:46835)
I20260929 02:19:44.674904 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:19:44.675138 17181 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.3:41295
W20260929 02:19:44.681077 17219 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:44.679749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46848 (local address 127.7.170.62:42449)
0929 02:19:44.679930 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.679936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.680269 (+ 333us) server_negotiation.cc:407] Connection header received
0929 02:19:44.680388 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.680392 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.680449 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.680527 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.680838 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46848: BlockingRecv error: recv got EOF from 127.0.0.1:46848 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:19:44.681670 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17240
I20260929 02:19:44.688629 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.682474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57704 (local address 127.7.170.1:34275)
0929 02:19:44.682951 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.682960 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.682989 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:19:44.683046 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.683052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.683227 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.683345 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.683643 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.684667 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.685954 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.686496 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.686588 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.687400 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:44.687407 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:44.687419 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:44.687954 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:44.687998 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:44.688020 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:44.688103 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:44.688182 (+ 79us) server_negotiation.cc:299] Negotiation successful
0929 02:19:44.688292 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":228,"threads_started":1}
I20260929 02:19:44.689775 17362 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:44.771456 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17240
I20260929 02:19:44.784163 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17370
I20260929 02:19:44.790954 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.784957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36646 (local address 127.7.170.2:35983)
0929 02:19:44.785420 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.785428 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.785454 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:19:44.785513 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.785519 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.785727 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.785889 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.786213 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.787135 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.788234 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.788768 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.788857 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.789779 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:44.789787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:44.789799 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:44.790355 (+ 556us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:44.790399 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:44.790417 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:44.790505 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:44.790581 (+ 76us) server_negotiation.cc:299] Negotiation successful
0929 02:19:44.790680 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":209,"threads_started":1}
I20260929 02:19:44.792095 17492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:44.870700 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17370
I20260929 02:19:44.882869 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17500
I20260929 02:19:44.890468 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:44.883724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45398 (local address 127.7.170.3:46835)
0929 02:19:44.884295 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:44.884302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:44.884330 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:19:44.884390 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:44.884397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:44.884619 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:44.884765 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:44.885230 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.886260 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.887475 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:44.888029 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:44.888139 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:44.889120 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:44.889129 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:44.889142 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:44.889703 (+ 561us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:44.889757 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:44.889776 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:44.889871 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:44.890014 (+ 143us) server_negotiation.cc:299] Negotiation successful
0929 02:19:44.890148 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":280,"threads_started":1}
I20260929 02:19:44.891626 17622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:44.969810 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17500
I20260929 02:19:44.982604 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17150
I20260929 02:19:44.983692 17211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:45.079425 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17150
2026-09-29T02:19:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2217 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-29T02:19:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:45Z Disabled control of system clock
I20260929 02:19:45.140772 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41739
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:41739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:19:45.455504 17646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:45.455771 17646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:45.455861 17646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:45.462697 17646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:45.462796 17646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:45.462839 17646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:45.462893 17646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:45.462935 17646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:45.471611 17646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:41739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:41739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:45.472787 17646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:45.474459 17646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:45.484763 17651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:45.484833 17652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:45.484803 17654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:45.485534 17646 server_base.cc:1034] running on GCE node
I20260929 02:19:45.486229 17646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:45.487489 17646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:45.488679 17646 hybrid_clock.cc:648] HybridClock initialized: now 1790648385488629 us; error 44 us; skew 500 ppm
I20260929 02:19:45.489025 17646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:45.491034 17646 webserver.cc:460] Webserver started at http://127.7.170.62:40883/ using document root <none> and password file <none>
I20260929 02:19:45.491679 17646 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:45.491770 17646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:45.492062 17646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:45.494757 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "21408571a3d741f3a45512f876f2cf53"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "b7ba0c6e10f7a88d77099f804c9c7273"
server_key_iv: "89206aab1b4bb9c2dbe23551090773a5"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.495514 17646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "21408571a3d741f3a45512f876f2cf53"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "b7ba0c6e10f7a88d77099f804c9c7273"
server_key_iv: "89206aab1b4bb9c2dbe23551090773a5"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.500155 17646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260929 02:19:45.503265 17660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:45.504447 17646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:45.504604 17646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "21408571a3d741f3a45512f876f2cf53"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "b7ba0c6e10f7a88d77099f804c9c7273"
server_key_iv: "89206aab1b4bb9c2dbe23551090773a5"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.504791 17646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:45.509876 17646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:45.510557 17646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:45.510803 17646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:45.526486 17646 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:41739
I20260929 02:19:45.526525 17711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:41739 every 8 connection(s)
I20260929 02:19:45.527658 17646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:45.531068 17712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:45.535995 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17646
I20260929 02:19:45.536197 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:45.536902 7848 external_mini_cluster.cc:1428] Setting key 9d9026443add82a75d23b5aa66b65859
I20260929 02:19:45.541754 17712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53: Bootstrap starting.
I20260929 02:19:45.544955 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:45.538960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.7.170.62:41739)
0929 02:19:45.539919 (+ 959us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:45.539930 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:45.539974 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:19:45.541507 (+ 1533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:45.541530 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:45.541924 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:45.542276 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:45.543331 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:45.543339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:45.543355 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:45.543373 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:45.543459 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:45.543481 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:45.543680 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:45.543805 (+ 125us) server_negotiation.cc:299] Negotiation successful
0929 02:19:45.544137 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":321,"threads_started":1}
I20260929 02:19:45.545044 17712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:45.546075 17712 log.cc:824] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:45.548578 17712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53: No bootstrap required, opened a new log
I20260929 02:19:45.555312 17712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } }
I20260929 02:19:45.555616 17712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:45.555706 17712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21408571a3d741f3a45512f876f2cf53, State: Initialized, Role: FOLLOWER
I20260929 02:19:45.556216 17712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } }
I20260929 02:19:45.556411 17712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:45.556535 17712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:45.556674 17712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:45.558903 17712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } }
I20260929 02:19:45.559345 17712 leader_election.cc:302] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21408571a3d741f3a45512f876f2cf53; no voters:
I20260929 02:19:45.559855 17712 leader_election.cc:288] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:45.559980 17717 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:45.560559 17717 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [term 1 LEADER]: Becoming Leader. State: Replica: 21408571a3d741f3a45512f876f2cf53, State: Running, Role: LEADER
I20260929 02:19:45.561107 17717 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } }
I20260929 02:19:45.561725 17712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:45.563524 17718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21408571a3d741f3a45512f876f2cf53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } } }
I20260929 02:19:45.563984 17718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:45.563524 17719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21408571a3d741f3a45512f876f2cf53. Latest consensus state: current_term: 1 leader_uuid: "21408571a3d741f3a45512f876f2cf53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21408571a3d741f3a45512f876f2cf53" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 41739 } } }
I20260929 02:19:45.565459 17719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:45.566516 17722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:45.571662 17722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:45.579794 17722 catalog_manager.cc:1347] Generated new cluster ID: 8e84e137f77040a99f3497e6de770ec1
I20260929 02:19:45.579886 17722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:45.599093 17722 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:45.599475 17722 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:45.615962 17722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21408571a3d741f3a45512f876f2cf53: Generated new TSK 0
I20260929 02:19:45.617065 17722 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:19:45.623750 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--builtin_ntp_servers=127.7.170.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260929 02:19:45.929239 17736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:45.929498 17736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:45.929620 17736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:45.937264 17736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:45.937363 17736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:45.937506 17736 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:45.946233 17736 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:45.947423 17736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:45.949151 17736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:45.960827 17742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:45.960966 17741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:45.960990 17744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:45.961817 17736 server_base.cc:1034] running on GCE node
I20260929 02:19:45.962385 17736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:45.963235 17736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:45.964416 17736 hybrid_clock.cc:648] HybridClock initialized: now 1790648385964362 us; error 42 us; skew 500 ppm
I20260929 02:19:45.964687 17736 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:45.966646 17736 webserver.cc:460] Webserver started at http://127.7.170.1:39389/ using document root <none> and password file <none>
I20260929 02:19:45.967303 17736 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:45.967396 17736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:45.967680 17736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:45.970373 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "9a71a62dd6314ed6ac225e44d204ba1b"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "86938865277176d871fc4cad1564f59f"
server_key_iv: "42f3ac2e547231032ed937bda8f056b8"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.971097 17736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "9a71a62dd6314ed6ac225e44d204ba1b"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "86938865277176d871fc4cad1564f59f"
server_key_iv: "42f3ac2e547231032ed937bda8f056b8"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.975679 17736 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:19:45.978762 17750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:45.979912 17736 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260929 02:19:45.980055 17736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "9a71a62dd6314ed6ac225e44d204ba1b"
format_stamp: "Formatted at 2026-09-29 02:19:45 on dist-test-slave-8cs2"
server_key: "86938865277176d871fc4cad1564f59f"
server_key_iv: "42f3ac2e547231032ed937bda8f056b8"
server_key_version: "encryptionkey@0"
I20260929 02:19:45.980232 17736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:45.985357 17736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:45.986054 17736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:45.986272 17736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:45.987051 17736 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:45.988265 17736 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:45.988353 17736 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:45.988449 17736 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:45.988512 17736 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.016321 17736 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:38717
I20260929 02:19:46.016364 17862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:38717 every 8 connection(s)
I20260929 02:19:46.017562 17736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:46.023253 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17736
I20260929 02:19:46.023434 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:46.023976 7848 external_mini_cluster.cc:1428] Setting key acb9a24f0d5b5cf25bd666873f4edfb5
I20260929 02:19:46.025094 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.019905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:37201 (local address 127.7.170.62:41739)
0929 02:19:46.020069 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.020074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.021408 (+ 1334us) server_negotiation.cc:407] Connection header received
0929 02:19:46.022993 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.022998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.023060 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.023144 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.024377 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:46.024383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:46.024389 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:46.024406 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:46.024440 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:46.024455 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:46.024587 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:46.024776 (+ 189us) server_negotiation.cc:299] Negotiation successful
0929 02:19:46.024824 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:19:46.026131 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.020370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:41739 (local address 127.7.170.1:37201)
0929 02:19:46.021264 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:46.021305 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:46.022736 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:46.023364 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:46.023375 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:46.023475 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:46.024089 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:46.024093 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:46.024139 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:46.024227 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:46.024567 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:46.024576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:46.024583 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:19:46.024839 (+ 256us) client_negotiation.cc:240] Negotiation successful
0929 02:19:46.025095 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":262,"threads_started":1}
I20260929 02:19:46.026753 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--builtin_ntp_servers=127.7.170.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 02:19:46.027456 17863 heartbeater.cc:347] Connected to a master server at 127.7.170.62:41739
I20260929 02:19:46.027715 17863 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:46.028221 17863 heartbeater.cc:511] Master 127.7.170.62:41739 requested a full tablet report, sending...
I20260929 02:19:46.030037 17677 ts_manager.cc:194] Registered new tserver with Master: 9a71a62dd6314ed6ac225e44d204ba1b (127.7.170.1:38717)
W20260929 02:19:46.329883 17866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:46.330137 17866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:46.330226 17866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:46.337180 17866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:46.337285 17866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:46.337504 17866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:46.347110 17866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:46.348280 17866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:46.349994 17866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:46.361703 17871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:46.361788 17872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:46.361913 17874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:46.362567 17866 server_base.cc:1034] running on GCE node
I20260929 02:19:46.363138 17866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:46.363973 17866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:46.365154 17866 hybrid_clock.cc:648] HybridClock initialized: now 1790648386365103 us; error 45 us; skew 500 ppm
I20260929 02:19:46.365432 17866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:46.367437 17866 webserver.cc:460] Webserver started at http://127.7.170.2:37505/ using document root <none> and password file <none>
I20260929 02:19:46.368046 17866 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:46.368139 17866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:46.368443 17866 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:46.371237 17866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "8dc843f6574c475894221f53efa1ee3f"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "6c37c577337a0f6c19b9a55834fff20f"
server_key_iv: "1b94b3c04af8e8d5bfeee893e1264756"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.372020 17866 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "8dc843f6574c475894221f53efa1ee3f"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "6c37c577337a0f6c19b9a55834fff20f"
server_key_iv: "1b94b3c04af8e8d5bfeee893e1264756"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.376619 17866 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260929 02:19:46.379691 17880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.380882 17866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260929 02:19:46.381053 17866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "8dc843f6574c475894221f53efa1ee3f"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "6c37c577337a0f6c19b9a55834fff20f"
server_key_iv: "1b94b3c04af8e8d5bfeee893e1264756"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.381230 17866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:46.386406 17866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:46.387100 17866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:46.387321 17866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:46.388134 17866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:46.389415 17866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:46.389510 17866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.389611 17866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:46.389678 17866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.417227 17866 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34723
I20260929 02:19:46.417280 17992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34723 every 8 connection(s)
I20260929 02:19:46.418685 17866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:46.425966 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.421064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:43633 (local address 127.7.170.62:41739)
0929 02:19:46.421263 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.421269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.422622 (+ 1353us) server_negotiation.cc:407] Connection header received
0929 02:19:46.424138 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.424143 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.424191 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.424273 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.425400 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:46.425405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:46.425408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:46.425422 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:46.425452 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:46.425463 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:46.425560 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:46.425677 (+ 117us) server_negotiation.cc:299] Negotiation successful
0929 02:19:46.425720 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:46.426110 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17866
I20260929 02:19:46.426283 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:46.426790 7848 external_mini_cluster.cc:1428] Setting key 461def5d1950254633938f721ed5d825
I20260929 02:19:46.427093 17995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.421635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:41739 (local address 127.7.170.2:43633)
0929 02:19:46.422482 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:46.422520 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:46.423908 (+ 1388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:46.424446 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:46.424453 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:46.424538 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:46.425139 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:46.425141 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:46.425175 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:46.425253 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:46.425534 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:46.425539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:46.425544 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:19:46.425731 (+ 187us) client_negotiation.cc:240] Negotiation successful
0929 02:19:46.425997 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":263,"threads_started":1}
I20260929 02:19:46.428373 17993 heartbeater.cc:347] Connected to a master server at 127.7.170.62:41739
I20260929 02:19:46.428642 17993 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:46.429302 17993 heartbeater.cc:511] Master 127.7.170.62:41739 requested a full tablet report, sending...
I20260929 02:19:46.429629 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--builtin_ntp_servers=127.7.170.20:33695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260929 02:19:46.430748 17677 ts_manager.cc:194] Registered new tserver with Master: 8dc843f6574c475894221f53efa1ee3f (127.7.170.2:34723)
W20260929 02:19:46.734830 17996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:46.735095 17996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:46.735226 17996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:46.743724 17996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:46.743826 17996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:46.743978 17996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:46.752806 17996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:41739
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:46.754037 17996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:46.755788 17996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:46.768034 18001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:46.768070 18002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:46.768091 18004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:46.768966 17996 server_base.cc:1034] running on GCE node
I20260929 02:19:46.769557 17996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:46.770375 17996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:46.771631 17996 hybrid_clock.cc:648] HybridClock initialized: now 1790648386771585 us; error 43 us; skew 500 ppm
I20260929 02:19:46.771914 17996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:19:46.774004 17996 webserver.cc:460] Webserver started at http://127.7.170.3:37993/ using document root <none> and password file <none>
I20260929 02:19:46.774616 17996 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:46.774711 17996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:46.775014 17996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:46.777661 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "a893edb3644841d385161537b021fa9c"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "d44cac5ece89c8d9fac128924fb1d934"
server_key_iv: "ff3d8d7ec6b21a102fb4a6c8679278db"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.778383 17996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "a893edb3644841d385161537b021fa9c"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "d44cac5ece89c8d9fac128924fb1d934"
server_key_iv: "ff3d8d7ec6b21a102fb4a6c8679278db"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.783021 17996 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:46.786249 18010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.787395 17996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s
I20260929 02:19:46.787537 17996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "a893edb3644841d385161537b021fa9c"
format_stamp: "Formatted at 2026-09-29 02:19:46 on dist-test-slave-8cs2"
server_key: "d44cac5ece89c8d9fac128924fb1d934"
server_key_iv: "ff3d8d7ec6b21a102fb4a6c8679278db"
server_key_version: "encryptionkey@0"
I20260929 02:19:46.787770 17996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:46.792819 17996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:46.793555 17996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:46.793761 17996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:46.794564 17996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:46.795795 17996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:46.795876 17996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.795951 17996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:46.796011 17996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:46.825067 17996 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:40499
I20260929 02:19:46.825106 18122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:40499 every 8 connection(s)
I20260929 02:19:46.826337 17996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:46.829058 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 17996
I20260929 02:19:46.829217 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:46.829636 7848 external_mini_cluster.cc:1428] Setting key fe668674e4a3e2f3d0eb02b8659bf31e
I20260929 02:19:46.834168 17715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.828773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35605 (local address 127.7.170.62:41739)
0929 02:19:46.829045 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.829051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.830598 (+ 1547us) server_negotiation.cc:407] Connection header received
0929 02:19:46.832178 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.832184 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.832246 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.832334 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.833529 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:46.833535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:46.833540 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:46.833556 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:46.833585 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:46.833596 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:46.833688 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:46.833859 (+ 171us) server_negotiation.cc:299] Negotiation successful
0929 02:19:46.833918 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260929 02:19:46.835057 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.829384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:41739 (local address 127.7.170.3:35605)
0929 02:19:46.830399 (+ 1015us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:46.830443 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:46.831925 (+ 1482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:46.832559 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:46.832567 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:46.832657 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:46.833241 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:19:46.833243 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:46.833279 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:19:46.833362 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:46.833676 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:46.833686 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:46.833693 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:19:46.833930 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:19:46.834198 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":304,"threads_started":1}
I20260929 02:19:46.836112 18123 heartbeater.cc:347] Connected to a master server at 127.7.170.62:41739
I20260929 02:19:46.836396 18123 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:46.836953 18123 heartbeater.cc:511] Master 127.7.170.62:41739 requested a full tablet report, sending...
I20260929 02:19:46.838171 17677 ts_manager.cc:194] Registered new tserver with Master: a893edb3644841d385161537b021fa9c (127.7.170.3:40499)
I20260929 02:19:46.844229 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260929 02:19:46.849722 17715 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:19:46.848360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.7.170.62:41739)
0929 02:19:46.848505 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.848509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.848985 (+ 476us) server_negotiation.cc:407] Connection header received
0929 02:19:46.849129 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.849132 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.849179 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.849245 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.849505 (+ 260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40980: BlockingRecv error: recv got EOF from 127.0.0.1:40980 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:19:46.850267 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17736
I20260929 02:19:46.853381 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.851126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.7.170.1:38717)
0929 02:19:46.851601 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.851607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.851631 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:19:46.851681 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.851686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.851856 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.851966 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.852725 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:46.852731 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:46.852741 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:46.852757 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:46.852793 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:46.852806 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:46.852901 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:46.853095 (+ 194us) server_negotiation.cc:299] Negotiation successful
0929 02:19:46.853169 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":234,"threads_started":1}
I20260929 02:19:46.854453 17858 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:46.931640 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17736
I20260929 02:19:46.944244 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17866
I20260929 02:19:46.947292 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:46.945079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.7.170.2:34723)
0929 02:19:46.945562 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:46.945569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:46.945595 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:19:46.945657 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:46.945663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:46.945904 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:46.946030 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:46.946770 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:46.946776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:46.946788 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:46.946799 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:46.946840 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:46.946857 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:46.946952 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:46.947003 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 02:19:46.947083 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":206,"threads_started":1}
I20260929 02:19:46.948441 17988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:47.026685 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17866
I20260929 02:19:47.038880 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17996
I20260929 02:19:47.042492 18136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:47.039839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36738 (local address 127.7.170.3:40499)
0929 02:19:47.040491 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:47.040498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:47.040527 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:19:47.040587 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:47.040594 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:47.040796 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:47.040919 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:47.041827 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:47.041833 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:47.041844 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:19:47.041860 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:47.041905 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:19:47.041919 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:47.042047 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:47.042098 (+ 51us) server_negotiation.cc:299] Negotiation successful
0929 02:19:47.042185 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":337,"threads_started":1}
I20260929 02:19:47.043642 18118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:47.122352 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17996
I20260929 02:19:47.134984 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 17646
I20260929 02:19:47.136058 17707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:47.233074 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 17646
2026-09-29T02:19:47Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2153 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:47 dist-test-slave-8cs2 krb5kdc[18141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:47 dist-test-slave-8cs2 krb5kdc[18141](info): set up 2 sockets
Sep 29 02:19:47 dist-test-slave-8cs2 krb5kdc[18141](info): commencing operation
krb5kdc: starting...
W20260929 02:19:48.241852 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.009s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:48 dist-test-slave-8cs2 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:19:48.448486 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42465
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:48.751677 18154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:48.751950 18154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:48.752084 18154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:48.759075 18154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:19:48.759184 18154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:48.759261 18154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:48.759344 18154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:19:48.759402 18154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:19:48.768882 18154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:48.770121 18154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:48.771339 18154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:48.781926 18159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:48.781893 18160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:48.781858 18162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:48.782716 18154 server_base.cc:1034] running on GCE node
I20260929 02:19:48.783421 18154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:48.784631 18154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:48.785852 18154 hybrid_clock.cc:648] HybridClock initialized: now 1790648388785774 us; error 26 us; skew 500 ppm
I20260929 02:19:48.786145 18154 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:48 dist-test-slave-8cs2 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:48.788913 18154 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:19:48.790217 18154 webserver.cc:460] Webserver started at http://127.7.170.62:37737/ using document root <none> and password file <none>
I20260929 02:19:48.790805 18154 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:48.790921 18154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:48.791188 18154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:48.793980 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "67f0da061e0e4f8092661cb64b03b356"
format_stamp: "Formatted at 2026-09-29 02:19:48 on dist-test-slave-8cs2"
server_key: "80aa61e83cfee8e18c588026fdbd5b1b"
server_key_iv: "09df6ce33f7b953520b4dac44c28d61b"
server_key_version: "encryptionkey@0"
I20260929 02:19:48.794647 18154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "67f0da061e0e4f8092661cb64b03b356"
format_stamp: "Formatted at 2026-09-29 02:19:48 on dist-test-slave-8cs2"
server_key: "80aa61e83cfee8e18c588026fdbd5b1b"
server_key_iv: "09df6ce33f7b953520b4dac44c28d61b"
server_key_version: "encryptionkey@0"
I20260929 02:19:48.799410 18154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:19:48.802445 18169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:48.803623 18154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:19:48.803753 18154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "67f0da061e0e4f8092661cb64b03b356"
format_stamp: "Formatted at 2026-09-29 02:19:48 on dist-test-slave-8cs2"
server_key: "80aa61e83cfee8e18c588026fdbd5b1b"
server_key_iv: "09df6ce33f7b953520b4dac44c28d61b"
server_key_version: "encryptionkey@0"
I20260929 02:19:48.803915 18154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:48.814375 18154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:48.818159 18154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:48.818388 18154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:48.834259 18154 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42465
I20260929 02:19:48.834282 18220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42465 every 8 connection(s)
I20260929 02:19:48.835493 18154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:19:48.838923 18221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:48.845311 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 18154
I20260929 02:19:48.845503 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:19:48.846081 7848 external_mini_cluster.cc:1428] Setting key aa804bc216d4c2cba672aa0cd7977131
I20260929 02:19:48.848623 18221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: Bootstrap starting.
I20260929 02:19:48.851733 18221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:48.852885 18221 log.cc:824] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:48.855829 18221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: No bootstrap required, opened a new log
Sep 29 02:19:48 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:48.864145 18221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } }
I20260929 02:19:48.864579 18221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:48.864688 18221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67f0da061e0e4f8092661cb64b03b356, State: Initialized, Role: FOLLOWER
I20260929 02:19:48.865283 18221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } }
I20260929 02:19:48.865489 18221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:19:48.865618 18221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:19:48.865761 18221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:48.867981 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:48.847788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.7.170.62:42465)
0929 02:19:48.848515 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:48.848526 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:48.848562 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:19:48.850175 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:48.850195 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:48.850552 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:48.850845 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:48.852535 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:48.854187 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:48.855890 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:48.856544 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:48.856660 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:48.861144 (+ 4484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:48.861151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:48.861168 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:48.861200 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:48.863963 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:48.864716 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:48.864723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:48.864728 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:48.864902 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:48.865294 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:48.865299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:48.865303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:48.865921 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:48.866223 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:48.866655 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 02:19:48.867052 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":319,"threads_started":1}
I20260929 02:19:48.868006 18221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } }
I20260929 02:19:48.868530 18221 leader_election.cc:302] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67f0da061e0e4f8092661cb64b03b356; no voters:
I20260929 02:19:48.869138 18221 leader_election.cc:288] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:19:48.869251 18226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:48.871328 18221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:19:48.875190 18226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [term 1 LEADER]: Becoming Leader. State: Replica: 67f0da061e0e4f8092661cb64b03b356, State: Running, Role: LEADER
I20260929 02:19:48.875783 18226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } }
I20260929 02:19:48.878392 18227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67f0da061e0e4f8092661cb64b03b356. Latest consensus state: current_term: 1 leader_uuid: "67f0da061e0e4f8092661cb64b03b356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } } }
I20260929 02:19:48.878561 18228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67f0da061e0e4f8092661cb64b03b356" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67f0da061e0e4f8092661cb64b03b356" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42465 } } }
I20260929 02:19:48.878859 18228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [sys.catalog]: This master's current role is: LEADER
I20260929 02:19:48.878757 18227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356 [sys.catalog]: This master's current role is: LEADER
W20260929 02:19:48.883801 18240 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:19:48.883909 18240 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:19:48.884037 18241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:19:48.886034 18241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:19:48.893657 18241 catalog_manager.cc:1347] Generated new cluster ID: 3197fbc43a7945d18332a2ee36d0a0b4
I20260929 02:19:48.893756 18241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:19:48.910939 18241 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:19:48.912125 18241 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:19:48.917675 18241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: Generated new TSK 0
I20260929 02:19:48.918418 18241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:19:48.972106 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:49.274401 18247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:49.274689 18247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:49.274788 18247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:49.281981 18247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:49.282097 18247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:49.282274 18247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:19:49.292059 18247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:49.293260 18247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:49.294497 18247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:49.306303 18253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:49.306416 18255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:49.306723 18252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:49.307174 18247 server_base.cc:1034] running on GCE node
I20260929 02:19:49.307793 18247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:49.308672 18247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:49.309903 18247 hybrid_clock.cc:648] HybridClock initialized: now 1790648389309828 us; error 62 us; skew 500 ppm
I20260929 02:19:49.310209 18247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:49 dist-test-slave-8cs2 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648389, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:49.313208 18247 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:19:49.314570 18247 webserver.cc:460] Webserver started at http://127.7.170.1:44441/ using document root <none> and password file <none>
I20260929 02:19:49.315203 18247 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:49.315316 18247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:49.315595 18247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:49.318423 18247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "a2622ed22f634d5ab21374bfb7c83d75"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "b7cd2ba5feb9985d8bcb70575f71832b"
server_key_iv: "5ba84d1b072c4bd317e57c48bdc2786e"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.319180 18247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "a2622ed22f634d5ab21374bfb7c83d75"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "b7cd2ba5feb9985d8bcb70575f71832b"
server_key_iv: "5ba84d1b072c4bd317e57c48bdc2786e"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.324218 18247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:19:49.327608 18262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.328979 18247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:49.329136 18247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "a2622ed22f634d5ab21374bfb7c83d75"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "b7cd2ba5feb9985d8bcb70575f71832b"
server_key_iv: "5ba84d1b072c4bd317e57c48bdc2786e"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.329317 18247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:49.347020 18247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:49.350551 18247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:49.350790 18247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:49.351645 18247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:49.352895 18247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:49.352990 18247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.353108 18247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:49.353160 18247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.382517 18247 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46795
I20260929 02:19:49.382562 18374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46795 every 8 connection(s)
I20260929 02:19:49.383690 18247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:19:49.390856 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 18247
I20260929 02:19:49.391047 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:19:49.391642 7848 external_mini_cluster.cc:1428] Setting key 9de7018fd493b277a1e15a7d755ba901
Sep 29 02:19:49 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648389, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:49.402347 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:49.385952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:36067 (local address 127.7.170.62:42465)
0929 02:19:49.386326 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:49.386330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:49.387286 (+ 956us) server_negotiation.cc:407] Connection header received
0929 02:19:49.388991 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:49.388997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:49.389069 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:49.389161 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:49.390527 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.391303 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.393077 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.393603 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.393738 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:49.396916 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:49.396922 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:49.396943 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:49.396977 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:49.399608 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:49.400363 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:49.400368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:49.400371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:49.400486 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:49.400869 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:49.400872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:49.400875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:49.401184 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:49.401309 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:49.401926 (+ 617us) server_negotiation.cc:299] Negotiation successful
0929 02:19:49.402102 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":186,"threads_started":1}
I20260929 02:19:49.403426 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:49.386506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42465 (local address 127.7.170.1:36067)
0929 02:19:49.387145 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:49.387182 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:49.388723 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:49.389390 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:49.389398 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:49.389859 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:49.390371 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:49.390387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.391433 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.391437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:49.392871 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:49.392884 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.393797 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.393802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:49.393989 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:49.394560 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:49.394586 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:49.396706 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:49.399740 (+ 3034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:49.399748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:49.399763 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:49.400271 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:49.400573 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:49.400577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:49.400580 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:49.400785 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:49.401297 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:49.401305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:49.401623 (+ 318us) client_negotiation.cc:769] Sending connection context
0929 02:19:49.401841 (+ 218us) client_negotiation.cc:240] Negotiation successful
0929 02:19:49.402207 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":241,"threads_started":1}
I20260929 02:19:49.404838 18375 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42465
I20260929 02:19:49.405182 18375 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:49.406039 18375 heartbeater.cc:511] Master 127.7.170.62:42465 requested a full tablet report, sending...
I20260929 02:19:49.408233 18186 ts_manager.cc:194] Registered new tserver with Master: a2622ed22f634d5ab21374bfb7c83d75 (127.7.170.1:46795)
I20260929 02:19:49.410122 18186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:36067
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:19:49.442267 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:49.752110 18381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:49.752398 18381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:49.752527 18381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:49.759550 18381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:49.759662 18381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:49.759804 18381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:19:49.769229 18381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:49.770438 18381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:49.771752 18381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:49.784065 18387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:49.784070 18389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:49.784029 18386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:49.785079 18381 server_base.cc:1034] running on GCE node
I20260929 02:19:49.785586 18381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:49.786437 18381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:49.787636 18381 hybrid_clock.cc:648] HybridClock initialized: now 1790648389787582 us; error 44 us; skew 500 ppm
I20260929 02:19:49.787917 18381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:49 dist-test-slave-8cs2 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648389, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:49.790874 18381 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:19:49.792225 18381 webserver.cc:460] Webserver started at http://127.7.170.2:42317/ using document root <none> and password file <none>
I20260929 02:19:49.792837 18381 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:49.792973 18381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:49.793256 18381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:49.795931 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "4631b7af29054e2ca806aa0ba8c47459"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "40552036c9a041269c91ae73950772ec"
server_key_iv: "f563219d0f3acc98f846bcf0092273f1"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.796653 18381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "4631b7af29054e2ca806aa0ba8c47459"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "40552036c9a041269c91ae73950772ec"
server_key_iv: "f563219d0f3acc98f846bcf0092273f1"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.801740 18381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:19:49.805369 18396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.806886 18381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:19:49.807042 18381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "4631b7af29054e2ca806aa0ba8c47459"
format_stamp: "Formatted at 2026-09-29 02:19:49 on dist-test-slave-8cs2"
server_key: "40552036c9a041269c91ae73950772ec"
server_key_iv: "f563219d0f3acc98f846bcf0092273f1"
server_key_version: "encryptionkey@0"
I20260929 02:19:49.807240 18381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:49.831595 18381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:49.835952 18381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:49.836225 18381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:49.837227 18381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:49.838711 18381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:49.838792 18381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.838910 18381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:49.838971 18381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:49.868338 18381 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:38435
I20260929 02:19:49.868391 18508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:38435 every 8 connection(s)
I20260929 02:19:49.869571 18381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:19:49.872244 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 18381
I20260929 02:19:49.872437 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:19:49.872905 7848 external_mini_cluster.cc:1428] Setting key 6a7f0a1ce38a6b0cb6bb8459bf2d58c6
Sep 29 02:19:49 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648389, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:49.889365 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:49.871889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:59787 (local address 127.7.170.62:42465)
0929 02:19:49.872090 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:49.872096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:49.873410 (+ 1314us) server_negotiation.cc:407] Connection header received
0929 02:19:49.875338 (+ 1928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:49.875342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:49.875404 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:49.875479 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:49.877283 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.877986 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.879912 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.880309 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.880402 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:49.884115 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:49.884120 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:49.884124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:49.884154 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:49.886619 (+ 2465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:49.887259 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:49.887263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:49.887265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:49.887377 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:49.887751 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:49.887756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:49.887760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:49.888035 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:49.888159 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:49.888810 (+ 651us) server_negotiation.cc:299] Negotiation successful
0929 02:19:49.889057 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:19:49.890381 18511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:49.872426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42465 (local address 127.7.170.2:59787)
0929 02:19:49.873232 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:49.873278 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:49.875068 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:49.875704 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:49.875714 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:49.876363 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:49.877127 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:49.877149 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.878128 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.878133 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:49.879777 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:49.879790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:49.880507 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:49.880513 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:49.880766 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:49.881424 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:49.881449 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:49.883898 (+ 2449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:49.886740 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:49.886748 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:49.886764 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:49.887142 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:49.887469 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:49.887474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:49.887478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:49.887644 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:49.888154 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:49.888162 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:49.888473 (+ 311us) client_negotiation.cc:769] Sending connection context
0929 02:19:49.888699 (+ 226us) client_negotiation.cc:240] Negotiation successful
0929 02:19:49.889223 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":269,"threads_started":1}
I20260929 02:19:49.891889 18509 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42465
I20260929 02:19:49.892181 18509 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:49.893003 18509 heartbeater.cc:511] Master 127.7.170.62:42465 requested a full tablet report, sending...
I20260929 02:19:49.894507 18186 ts_manager.cc:194] Registered new tserver with Master: 4631b7af29054e2ca806aa0ba8c47459 (127.7.170.2:38435)
I20260929 02:19:49.895638 18186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:59787
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:19:49.923978 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:37923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:19:50.226672 18514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:19:50.226927 18514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:19:50.227026 18514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:19:50.233850 18514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:19:50.233945 18514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:19:50.234058 18514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:19:50.243275 18514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:37923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:19:50.244510 18514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:19:50.245777 18514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:19:50.257871 18520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:50.257886 18522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:19:50.257882 18519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:19:50.258728 18514 server_base.cc:1034] running on GCE node
I20260929 02:19:50.259290 18514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:19:50.260130 18514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:19:50.261312 18514 hybrid_clock.cc:648] HybridClock initialized: now 1790648390261255 us; error 50 us; skew 500 ppm
I20260929 02:19:50.261601 18514 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:19:50 dist-test-slave-8cs2 krb5kdc[18141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648390, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:19:50.269562 18514 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:19:50.270849 18514 webserver.cc:460] Webserver started at http://127.7.170.3:40769/ using document root <none> and password file <none>
I20260929 02:19:50.271522 18514 fs_manager.cc:359] Metadata directory not provided
I20260929 02:19:50.271618 18514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:19:50.271881 18514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:19:50.274835 18514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "67e7821c5d6545a5b9508bbfae9b4177"
format_stamp: "Formatted at 2026-09-29 02:19:50 on dist-test-slave-8cs2"
server_key: "e94ffd0d24bfebfc9c703149a55674d5"
server_key_iv: "14ae0fde40283d02a35631446a47ba19"
server_key_version: "encryptionkey@0"
I20260929 02:19:50.275559 18514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "67e7821c5d6545a5b9508bbfae9b4177"
format_stamp: "Formatted at 2026-09-29 02:19:50 on dist-test-slave-8cs2"
server_key: "e94ffd0d24bfebfc9c703149a55674d5"
server_key_iv: "14ae0fde40283d02a35631446a47ba19"
server_key_version: "encryptionkey@0"
I20260929 02:19:50.280321 18514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:19:50.283514 18529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:50.285467 18514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:19:50.285626 18514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "67e7821c5d6545a5b9508bbfae9b4177"
format_stamp: "Formatted at 2026-09-29 02:19:50 on dist-test-slave-8cs2"
server_key: "e94ffd0d24bfebfc9c703149a55674d5"
server_key_iv: "14ae0fde40283d02a35631446a47ba19"
server_key_version: "encryptionkey@0"
I20260929 02:19:50.285887 18514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:19:50.309558 18514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:19:50.313582 18514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:19:50.313831 18514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:19:50.314697 18514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:19:50.315959 18514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:19:50.316025 18514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:50.316109 18514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:19:50.316154 18514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:19:50.344964 18514 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:34007
I20260929 02:19:50.345008 18641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:34007 every 8 connection(s)
I20260929 02:19:50.346226 18514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:19:50.353826 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 18514
I20260929 02:19:50.354027 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:19:50.354497 7848 external_mini_cluster.cc:1428] Setting key c365d7270e95c1d6b65a1b638f7c5eff
Sep 29 02:19:50 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648390, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:19:50.364769 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.348533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:57571 (local address 127.7.170.62:42465)
0929 02:19:50.348706 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.348710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.349876 (+ 1166us) server_negotiation.cc:407] Connection header received
0929 02:19:50.351622 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.351627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.351692 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.351776 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:50.353255 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.354212 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.355722 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.356287 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.356393 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.359825 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:50.359830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:50.359833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:50.359866 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:50.362314 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.362967 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.362971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.362983 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.363102 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.363430 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.363432 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.363435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.363683 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:50.363784 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.364374 (+ 590us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.364528 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:19:50.365809 18644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.349038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42465 (local address 127.7.170.3:57571)
0929 02:19:50.349719 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.349757 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.351369 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.351980 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.352002 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.352513 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:19:50.353120 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.353139 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.354348 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.354352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.355598 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.355610 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.356505 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.356513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.356792 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.357514 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:19:50.357536 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:19:50.359612 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:19:50.362429 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:50.362436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:50.362448 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:50.362845 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:50.363181 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:19:50.363184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:19:50.363186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:19:50.363353 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:19:50.363772 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:19:50.363778 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:19:50.364091 (+ 313us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.364304 (+ 213us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.364679 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":256,"threads_started":1}
I20260929 02:19:50.367095 18642 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42465
I20260929 02:19:50.367368 18642 heartbeater.cc:464] Registering TS with master...
I20260929 02:19:50.368062 18642 heartbeater.cc:511] Master 127.7.170.62:42465 requested a full tablet report, sending...
I20260929 02:19:50.369454 18185 ts_manager.cc:194] Registered new tserver with Master: 67e7821c5d6545a5b9508bbfae9b4177 (127.7.170.3:34007)
I20260929 02:19:50.370283 18185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:57571
I20260929 02:19:50.381261 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:19:50.397480 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.386268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.7.170.62:42465)
0929 02:19:50.386427 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.386433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.386914 (+ 481us) server_negotiation.cc:407] Connection header received
0929 02:19:50.387032 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.387037 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.387108 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.387204 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:50.388437 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.389171 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.390359 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.390785 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.390886 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.392997 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:50.393003 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:50.393008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:50.393039 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:50.395109 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.395686 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.395692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.395696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.395826 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.396174 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.396179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.396182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.396457 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:50.396671 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.397020 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.397213 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:19:50.401007 18185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:19:50.407631 18185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:19:50.413203 18375 heartbeater.cc:503] Master 127.7.170.62:42465 was elected leader, sending a full tablet report...
I20260929 02:19:50.428759 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.418462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:38435 (local address 127.0.0.1:41752)
0929 02:19:50.419268 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.419309 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.419476 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.420510 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.420519 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.420551 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:50.421006 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.421018 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.423048 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.423054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.425548 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.425565 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.427772 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.427779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.428003 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.428053 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.428181 (+ 128us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.428373 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":555,"threads_started":1}
I20260929 02:19:50.429783 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.418564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.7.170.2:38435)
0929 02:19:50.419096 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.419105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.419648 (+ 543us) server_negotiation.cc:407] Connection header received
0929 02:19:50.419750 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.419758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.420041 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.420232 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:50.421202 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.422867 (+ 1665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.425745 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.427287 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.427413 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.427621 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.429258 (+ 1637us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.429443 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":754,"threads_started":1}
I20260929 02:19:50.434888 18444 tablet_service.cc:1467] Processing CreateTablet for tablet f9e9b2cbb53340c4ab525828fae31776 (DEFAULT_TABLE table=test-table [id=759c5de191f84a209c3908e689c14817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:50.436417 18444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e9b2cbb53340c4ab525828fae31776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:50.438828 18657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.419021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:34007 (local address 127.0.0.1:52892)
0929 02:19:50.430980 (+ 11959us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.431003 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.431107 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.432025 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.432030 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.432055 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:50.432383 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.432394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.434295 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.434299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.436179 (+ 1880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.436193 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.437885 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.437892 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.438141 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.438178 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.438274 (+ 96us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.438396 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11834,"mutex_wait_us":847,"thread_start_us":15414,"threads_started":1}
I20260929 02:19:50.439489 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.419209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52892 (local address 127.7.170.3:34007)
0929 02:19:50.419745 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.419753 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.431222 (+ 11469us) server_negotiation.cc:407] Connection header received
0929 02:19:50.431322 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.431328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.431571 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.431749 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:50.432580 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.434146 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.436354 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.437680 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.437805 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.438046 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.439037 (+ 991us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.439202 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":332,"threads_started":1}
I20260929 02:19:50.441701 18654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.419767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46795 (local address 127.0.0.1:38894)
0929 02:19:50.430199 (+ 10432us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.430219 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.430320 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.435105 (+ 4785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.435109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.435132 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:50.435340 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.435355 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.437091 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.437094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.438656 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.438676 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.440959 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.440965 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.441154 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.441184 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.441266 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.441393 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10336,"mutex_wait_us":22,"thread_start_us":10961,"threads_started":1}
I20260929 02:19:50.441745 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.419961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.7.170.1:46795)
0929 02:19:50.434688 (+ 14727us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.434694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.434715 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:19:50.434781 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.434785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.434936 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.435094 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:50.435537 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.436917 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.438846 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.440796 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.440908 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.441149 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.441316 (+ 167us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.441462 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14598,"thread_start_us":14435,"threads_started":1}
I20260929 02:19:50.442941 18577 tablet_service.cc:1467] Processing CreateTablet for tablet f9e9b2cbb53340c4ab525828fae31776 (DEFAULT_TABLE table=test-table [id=759c5de191f84a209c3908e689c14817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:50.444320 18577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e9b2cbb53340c4ab525828fae31776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:50.444555 18310 tablet_service.cc:1467] Processing CreateTablet for tablet f9e9b2cbb53340c4ab525828fae31776 (DEFAULT_TABLE table=test-table [id=759c5de191f84a209c3908e689c14817]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:19:50.445562 18310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9e9b2cbb53340c4ab525828fae31776. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:19:50.450277 18660 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Bootstrap starting.
I20260929 02:19:50.453855 18660 tablet_bootstrap.cc:654] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:50.455070 18660 log.cc:824] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:50.457432 18662 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Bootstrap starting.
I20260929 02:19:50.459304 18660 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: No bootstrap required, opened a new log
I20260929 02:19:50.459574 18660 ts_tablet_manager.cc:1397] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260929 02:19:50.460103 18663 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Bootstrap starting.
I20260929 02:19:50.461536 18662 tablet_bootstrap.cc:654] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:50.462450 18662 log.cc:824] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:50.463402 18663 tablet_bootstrap.cc:654] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Neither blocks nor log segments found. Creating new log.
I20260929 02:19:50.464574 18663 log.cc:824] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Log is configured to *not* fsync() on all Append() calls
I20260929 02:19:50.464718 18662 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: No bootstrap required, opened a new log
I20260929 02:19:50.464972 18662 ts_tablet_manager.cc:1397] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260929 02:19:50.467904 18663 tablet_bootstrap.cc:492] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: No bootstrap required, opened a new log
I20260929 02:19:50.468168 18663 ts_tablet_manager.cc:1397] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260929 02:19:50.468362 18660 raft_consensus.cc:348] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.468722 18660 raft_consensus.cc:374] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:50.468842 18660 raft_consensus.cc:729] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4631b7af29054e2ca806aa0ba8c47459, State: Initialized, Role: FOLLOWER
I20260929 02:19:50.469425 18660 consensus_queue.cc:260] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.470387 18509 heartbeater.cc:503] Master 127.7.170.62:42465 was elected leader, sending a full tablet report...
I20260929 02:19:50.471040 18660 ts_tablet_manager.cc:1428] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260929 02:19:50.474993 18662 raft_consensus.cc:348] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.475401 18662 raft_consensus.cc:374] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:50.475497 18662 raft_consensus.cc:729] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2622ed22f634d5ab21374bfb7c83d75, State: Initialized, Role: FOLLOWER
I20260929 02:19:50.476063 18662 consensus_queue.cc:260] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.477691 18662 ts_tablet_manager.cc:1428] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Time spent starting tablet: real 0.013s user 0.005s sys 0.007s
I20260929 02:19:50.477973 18663 raft_consensus.cc:348] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.478286 18663 raft_consensus.cc:374] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:19:50.478364 18663 raft_consensus.cc:729] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67e7821c5d6545a5b9508bbfae9b4177, State: Initialized, Role: FOLLOWER
I20260929 02:19:50.478811 18663 consensus_queue.cc:260] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.479743 18642 heartbeater.cc:503] Master 127.7.170.62:42465 was elected leader, sending a full tablet report...
I20260929 02:19:50.480160 18663 ts_tablet_manager.cc:1428] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Time spent starting tablet: real 0.012s user 0.007s sys 0.006s
I20260929 02:19:50.539873 18666 raft_consensus.cc:482] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:19:50.540138 18666 raft_consensus.cc:504] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.541788 18666 leader_election.cc:288] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2622ed22f634d5ab21374bfb7c83d75 (127.7.170.1:46795), 67e7821c5d6545a5b9508bbfae9b4177 (127.7.170.3:34007)
I20260929 02:19:50.549249 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.542313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:37293 (local address 127.7.170.1:46795)
0929 02:19:50.542485 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.542491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.542876 (+ 385us) server_negotiation.cc:407] Connection header received
0929 02:19:50.543098 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.543104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.543185 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.543273 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:50.543822 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.544828 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.547234 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.548364 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.548502 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.548599 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.548872 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.549012 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:19:50.549417 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.542148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42303 (local address 127.7.170.3:34007)
0929 02:19:50.542347 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.542353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.543375 (+ 1022us) server_negotiation.cc:407] Connection header received
0929 02:19:50.543511 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.543515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.543571 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.543635 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:19:50.544276 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.545206 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.547402 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.548560 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.548666 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.548734 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.549071 (+ 337us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.549194 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:19:50.549638 18669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.542013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46795 (local address 127.7.170.2:37293)
0929 02:19:50.542775 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.542790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.542917 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.543313 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.543317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.543345 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:50.543681 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.543692 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.544961 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.544967 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.547093 (+ 2126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.547107 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.548558 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.548563 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.548720 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.548755 (+ 35us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.548852 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.548998 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":486,"threads_started":1}
I20260929 02:19:50.549638 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.542014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:34007 (local address 127.7.170.2:42303)
0929 02:19:50.543268 (+ 1254us) negotiation.cc:107] Waiting for socket to connect
0929 02:19:50.543283 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:19:50.543396 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:19:50.543681 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:19:50.543685 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:19:50.543708 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:19:50.544069 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.544082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.545321 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.545326 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.547283 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:19:50.547292 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.548738 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.548744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:19:50.548903 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.548962 (+ 59us) client_negotiation.cc:769] Sending connection context
0929 02:19:50.549044 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:19:50.549156 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"mutex_wait_us":33,"thread_start_us":974,"threads_started":1}
I20260929 02:19:50.550470 18597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e9b2cbb53340c4ab525828fae31776" candidate_uuid: "4631b7af29054e2ca806aa0ba8c47459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e7821c5d6545a5b9508bbfae9b4177" is_pre_election: true
I20260929 02:19:50.550602 18330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e9b2cbb53340c4ab525828fae31776" candidate_uuid: "4631b7af29054e2ca806aa0ba8c47459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2622ed22f634d5ab21374bfb7c83d75" is_pre_election: true
I20260929 02:19:50.550961 18597 raft_consensus.cc:2454] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4631b7af29054e2ca806aa0ba8c47459 in term 0.
I20260929 02:19:50.551056 18330 raft_consensus.cc:2454] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4631b7af29054e2ca806aa0ba8c47459 in term 0.
I20260929 02:19:50.551527 18399 leader_election.cc:302] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4631b7af29054e2ca806aa0ba8c47459, 67e7821c5d6545a5b9508bbfae9b4177; no voters:
I20260929 02:19:50.551851 18666 raft_consensus.cc:2781] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:19:50.551994 18666 raft_consensus.cc:482] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:19:50.552093 18666 raft_consensus.cc:3037] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:50.554183 18666 raft_consensus.cc:504] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.554698 18666 leader_election.cc:288] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [CANDIDATE]: Term 1 election: Requested vote from peers a2622ed22f634d5ab21374bfb7c83d75 (127.7.170.1:46795), 67e7821c5d6545a5b9508bbfae9b4177 (127.7.170.3:34007)
I20260929 02:19:50.555047 18330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e9b2cbb53340c4ab525828fae31776" candidate_uuid: "4631b7af29054e2ca806aa0ba8c47459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2622ed22f634d5ab21374bfb7c83d75"
I20260929 02:19:50.555148 18597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9e9b2cbb53340c4ab525828fae31776" candidate_uuid: "4631b7af29054e2ca806aa0ba8c47459" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "67e7821c5d6545a5b9508bbfae9b4177"
I20260929 02:19:50.555204 18330 raft_consensus.cc:3037] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:50.555321 18597 raft_consensus.cc:3037] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:19:50.557399 18330 raft_consensus.cc:2454] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4631b7af29054e2ca806aa0ba8c47459 in term 1.
I20260929 02:19:50.557699 18597 raft_consensus.cc:2454] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4631b7af29054e2ca806aa0ba8c47459 in term 1.
I20260929 02:19:50.557708 18397 leader_election.cc:302] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4631b7af29054e2ca806aa0ba8c47459, a2622ed22f634d5ab21374bfb7c83d75; no voters:
I20260929 02:19:50.557997 18666 raft_consensus.cc:2781] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:19:50.558393 18666 raft_consensus.cc:686] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 1 LEADER]: Becoming Leader. State: Replica: 4631b7af29054e2ca806aa0ba8c47459, State: Running, Role: LEADER
I20260929 02:19:50.558889 18666 consensus_queue.cc:237] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } }
I20260929 02:19:50.563081 18186 catalog_manager.cc:5495] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4631b7af29054e2ca806aa0ba8c47459 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "4631b7af29054e2ca806aa0ba8c47459" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4631b7af29054e2ca806aa0ba8c47459" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 38435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 } health_report { overall_health: UNKNOWN } } }
W20260929 02:19:50.597676 18643 tablet.cc:2369] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:50.603726 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.599354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.7.170.2:38435)
0929 02:19:50.599521 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.599525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.599542 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:19:50.599619 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.599622 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.599698 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.599809 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:19:50.600080 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.600756 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.601986 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.602356 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.602443 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.602782 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:19:50.603281 (+ 499us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:19:50.603383 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.603422 (+ 39us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.603479 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:19:50.611281 18597 raft_consensus.cc:1261] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 1 FOLLOWER]: Refusing update from remote peer 4631b7af29054e2ca806aa0ba8c47459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:50.611285 18330 raft_consensus.cc:1261] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 1 FOLLOWER]: Refusing update from remote peer 4631b7af29054e2ca806aa0ba8c47459: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:19:50.612160 18671 consensus_queue.cc:1035] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67e7821c5d6545a5b9508bbfae9b4177" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:19:50.612461 18666 consensus_queue.cc:1035] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2622ed22f634d5ab21374bfb7c83d75" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260929 02:19:50.621415 18510 tablet.cc:2369] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:19:50.622287 18674 mvcc.cc:205] Tried to move back new op lower bound from 7334495807935135744 to 7334495807732944896. Current Snapshot: MvccSnapshot[applied={T|T < 7334495807935135744}]
I20260929 02:19:50.626128 18675 mvcc.cc:205] Tried to move back new op lower bound from 7334495807935135744 to 7334495807732944896. Current Snapshot: MvccSnapshot[applied={T|T < 7334495807935135744}]
I20260929 02:19:50.636169 18185 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:50.636422 18185 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:19:50.639664 18185 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67f0da061e0e4f8092661cb64b03b356: Sending DeleteTablet for 3 replicas of tablet f9e9b2cbb53340c4ab525828fae31776
I20260929 02:19:50.640585 18444 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e9b2cbb53340c4ab525828fae31776 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:50 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:41752
I20260929 02:19:50.640686 18310 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e9b2cbb53340c4ab525828fae31776 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:50 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:38894
I20260929 02:19:50.640971 18577 tablet_service.cc:1514] Processing DeleteTablet for tablet f9e9b2cbb53340c4ab525828fae31776 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:19:50 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:52892
I20260929 02:19:50.642236 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 18247
I20260929 02:19:50.659979 18683 tablet_replica.cc:330] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: stopping tablet replica
I20260929 02:19:50.662453 18683 raft_consensus.cc:2229] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 1 LEADER]: Raft consensus shutting down.
Sep 29 02:19:50 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:19:50.666778 18683 raft_consensus.cc:2258] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:50.667690 18684 tablet_replica.cc:330] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: stopping tablet replica
I20260929 02:19:50.667903 18685 tablet_replica.cc:330] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: stopping tablet replica
I20260929 02:19:50.668289 18685 raft_consensus.cc:2229] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:50.668761 18685 raft_consensus.cc:2258] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:50.669354 18684 raft_consensus.cc:2229] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:19:50.669752 18683 ts_tablet_manager.cc:1905] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:50.669850 18684 raft_consensus.cc:2258] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:19:50.671706 18685 ts_tablet_manager.cc:1905] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:50.673022 18684 ts_tablet_manager.cc:1905] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:19:50.677223 18683 ts_tablet_manager.cc:1918] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:50.677448 18683 log.cc:1196] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/f9e9b2cbb53340c4ab525828fae31776
I20260929 02:19:50.678056 18683 ts_tablet_manager.cc:1939] T f9e9b2cbb53340c4ab525828fae31776 P 4631b7af29054e2ca806aa0ba8c47459: Deleting consensus metadata
I20260929 02:19:50.678364 18685 ts_tablet_manager.cc:1918] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:50.678507 18685 log.cc:1196] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/f9e9b2cbb53340c4ab525828fae31776
I20260929 02:19:50.678972 18685 ts_tablet_manager.cc:1939] T f9e9b2cbb53340c4ab525828fae31776 P 67e7821c5d6545a5b9508bbfae9b4177: Deleting consensus metadata
I20260929 02:19:50.679466 18684 ts_tablet_manager.cc:1918] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:19:50.679620 18684 log.cc:1196] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/f9e9b2cbb53340c4ab525828fae31776
I20260929 02:19:50.680073 18684 ts_tablet_manager.cc:1939] T f9e9b2cbb53340c4ab525828fae31776 P a2622ed22f634d5ab21374bfb7c83d75: Deleting consensus metadata
I20260929 02:19:50.680164 18170 catalog_manager.cc:4841] TS 4631b7af29054e2ca806aa0ba8c47459 (127.7.170.2:38435): tablet f9e9b2cbb53340c4ab525828fae31776 (table test-table [id=759c5de191f84a209c3908e689c14817]) successfully deleted
I20260929 02:19:50.681022 18172 catalog_manager.cc:4841] TS 67e7821c5d6545a5b9508bbfae9b4177 (127.7.170.3:34007): tablet f9e9b2cbb53340c4ab525828fae31776 (table test-table [id=759c5de191f84a209c3908e689c14817]) successfully deleted
I20260929 02:19:50.681730 18170 catalog_manager.cc:4841] TS a2622ed22f634d5ab21374bfb7c83d75 (127.7.170.1:46795): tablet f9e9b2cbb53340c4ab525828fae31776 (table test-table [id=759c5de191f84a209c3908e689c14817]) successfully deleted
I20260929 02:19:50.681945 18659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.643739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.7.170.1:46795)
0929 02:19:50.643911 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.643918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.653861 (+ 9943us) server_negotiation.cc:407] Connection header received
0929 02:19:50.654274 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.654281 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.654394 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.654466 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:50.656248 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.657522 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.660194 (+ 2672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.660898 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.661009 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.669518 (+ 8509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:50.669534 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:50.669555 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:50.669595 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:50.672952 (+ 3357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.674927 (+ 1975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.674940 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.674943 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.675111 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.675562 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.675568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.675572 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.676025 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:50.676305 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.681390 (+ 5085us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.681607 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:19:50.682512 18370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:50.772954 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 18247
I20260929 02:19:50.787320 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 18381
Sep 29 02:19:50 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:19:50.802946 18655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.788200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41776 (local address 127.7.170.2:38435)
0929 02:19:50.788403 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.788409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.788431 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:19:50.788509 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.788515 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.788595 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.788706 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:50.789976 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.791206 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.792360 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.793057 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.793196 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.796768 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:50.796785 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:50.796810 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:50.796854 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:50.800427 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.801006 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.801015 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.801018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.801183 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.801490 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.801495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.801499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.801890 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:50.802200 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.802340 (+ 140us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.802557 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:19:50.803694 18504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:50.914938 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 18381
I20260929 02:19:50.929594 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 18514
Sep 29 02:19:50 dist-test-slave-8cs2 krb5kdc[18141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:19:50.944806 18656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:19:50.930251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.7.170.3:34007)
0929 02:19:50.930394 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0929 02:19:50.930399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:19:50.930415 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:19:50.930548 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:19:50.930551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:19:50.930615 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:19:50.930688 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:19:50.931784 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.932479 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.933646 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:19:50.934026 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:19:50.934113 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:19:50.938800 (+ 4687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:19:50.938820 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:19:50.938840 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:19:50.938885 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:19:50.942229 (+ 3344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.942825 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.942836 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.942840 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.942971 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:19:50.943316 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:19:50.943321 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:19:50.943325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:19:50.943683 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:19:50.943942 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0929 02:19:50.944238 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 02:19:50.944443 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:19:50.945436 18637 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:51.032310 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 18514
I20260929 02:19:51.045672 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 18154
I20260929 02:19:51.046743 18216 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:19:51.149905 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 18154
2026-09-29T02:19:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (3918 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:19:51 dist-test-slave-8cs2 krb5kdc[18693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:19:51 dist-test-slave-8cs2 krb5kdc[18693](info): set up 2 sockets
Sep 29 02:19:51 dist-test-slave-8cs2 krb5kdc[18693](info): commencing operation
krb5kdc: starting...
W20260929 02:19:52.149793 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.963s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:19:52 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648392, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:19:52Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangeradmin_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangeradmin_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangerlookup_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangerlookup_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
I20260929 02:19:52.429921 7848 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoo3C6V/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/postgres -l logfile start
2026-09-29 02:19:54.592 UTC [18723] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 02:19:54.592 UTC [18723] LOG: listening on IPv4 address "127.7.170.20", port 35897
2026-09-29 02:19:54.594 UTC [18723] LOG: listening on Unix socket "/tmp/.s.PGSQL.35897"
2026-09-29 02:19:54.614 UTC [18726] LOG: database system was shut down at 2026-09-29 02:19:53 UTC
2026-09-29 02:19:54.617 UTC [18723] LOG: database system is ready to accept connections
I20260929 02:19:55.509634 7848 mini_postgres.cc:96] Postgres bound to 35897
2026-09-29 02:19:55.526 UTC [18734] FATAL: database "slave" does not exist
127.7.170.20:35897 - accepting connections
I20260929 02:19:55.527444 7848 mini_ranger.cc:161] Starting Ranger...
I20260929 02:19:55.551978 7848 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 02:19:55.978940 7848 mini_ranger.cc:87] Created ranger Postgres database
I20260929 02:19:55.979066 7848 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin
2026-09-29 02:19:56,254 [I] DB FLAVOR :POSTGRES
2026-09-29 02:19:56,255 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:19:56,255 [I] Checking connection..
2026-09-29 02:19:56,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 02:19:56,483 [I] Checking connection passed.
2026-09-29 02:19:56,483 [I] --------- Verifying version history table ---------
2026-09-29 02:19:56,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:19:56,739 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:19:56,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:19:56,973 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:19:56,973 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 02:19:56,973 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 02:19:57.215 UTC [18798] WARNING: there is no transaction in progress
2026-09-29 02:19:57,220 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 02:19:57,220 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:19:57,466 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 02:19:57,467 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:19:57,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 02:19:57,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 02:19:57,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8cs2.c.gcp-upstream.internal','N') ;"
2026-09-29 02:19:58,152 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:19:58,390 [I] Table x_portal_user does not exist in database ranger
2026-09-29 02:19:58,390 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:19:58,635 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 02:19:58,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:19:58,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:19:59,126 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 02:19:59,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 02:19:59.997 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.012 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.027 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.041 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.054 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.070 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.120 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.125 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.137 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.146 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.155 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.167 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.175 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.181 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.187 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00.192 UTC [18906] WARNING: there is no transaction in progress
2026-09-29 02:20:00,727 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 02:20:00,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 02:20:00,968 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 02:20:00,968 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:20:01,211 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 02:20:01,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:20:01,442 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 02:20:01,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:20:01,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:20:01,892 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8cs2.c.gcp-upstream.internal';"
2026-09-29 02:20:02,137 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 02:20:02,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 02:20:02,389 [I] DB_PATCHES have already been applied
I20260929 02:20:02.395047 7848 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoo3C6V/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 02:20:02.395133 7848 mini_ranger.cc:193] Using host: 127.7.170.20
I20260929 02:20:02.410284 7848 mini_ranger.cc:227] Ranger admin URL: http://127.7.170.20:47593
Sep 29, 2026 2:20:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:20:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:20:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 2:20:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 2:20:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 2:20:03 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 2:20:03 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 02:20:03.452382 7848 mini_ranger.cc:160] Time spent starting Ranger: real 7.925s user 0.003s sys 0.040s
Sep 29, 2026 2:20:21 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 2:20:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 29, 2026 2:20:21 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 2:20:21 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 2:20:22 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 02:20:23.090662 7848 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangerkms_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/rangerkms_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.170.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.170.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 02:20:23 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648423, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 02:20:23.234524 7848 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 02:20:23.253250 7848 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 02:20:24.003320 7848 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 02:20:24.003464 7848 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms
I20260929 02:20:24.003609 7848 mini_ranger_kms.cc:227] Using postgres at 127.7.170.20:35897
2026-09-29 02:20:24,411 [I] DB FLAVOR :POSTGRES
2026-09-29 02:20:24,412 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:20:24,412 [I] Checking connection
2026-09-29 02:20:24,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 02:20:24,646 [I] connection success
2026-09-29 02:20:24,646 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 02:20:24,646 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 02:20:24,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 02:20:24,884 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 02:20:24,884 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:20:24,885 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 02:20:24,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:35897/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 02:20:25,144 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 02:20:25.620043 7848 mini_ranger_kms.cc:310] Created kms service
I20260929 02:20:25.817792 7848 mini_ranger_kms.cc:326] Created kudu user
I20260929 02:20:25.858824 7848 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 02:20:25 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648423, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:26.240825 7848 mini_ranger_kms.cc:384] Added ranger policy
I20260929 02:20:26.241232 7848 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 02:20:26.241302 7848 mini_ranger_kms.cc:240] Using host: 127.7.170.20
I20260929 02:20:26.263707 7848 mini_ranger_kms.cc:276] Ranger KMS PID: 19086
I20260929 02:20:26.263862 7848 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.170.20:52539
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 2:20:27 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 02:20:28.301287 7848 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.067s user 0.115s sys 0.316s
I20260929 02:20:28.301527 7848 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 02:20:28.301728 7848 mini_ranger_kms.cc:401] 127.7.170.20:52539/kms/v1/keys
Sep 29 02:20:28 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790648428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 02:20:28 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
Sep 29 02:20:29 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648428, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
Sep 29 02:20:31 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:20:31.371954 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42769
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42769
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:52539/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:20:31.701682 19137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:20:31.701961 19137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:20:31.702056 19137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:20:31.709355 19137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:20:31.709447 19137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:20:31.709498 19137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:20:31.709550 19137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:20:31.709594 19137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:20:31.719187 19137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.170.20:52539/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42769
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:20:31.720357 19137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:20:31.721680 19137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:20:31.733244 19145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:31.733285 19143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:31.733306 19142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:20:31.734314 19137 server_base.cc:1034] running on GCE node
I20260929 02:20:31.734975 19137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:20:31.736294 19137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:20:31.737473 19137 hybrid_clock.cc:648] HybridClock initialized: now 1790648431737434 us; error 33 us; skew 500 ppm
I20260929 02:20:31.737790 19137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:20:31 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:20:31.745743 19137 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:20:31.747171 19137 webserver.cc:460] Webserver started at http://127.7.170.62:41903/ using document root <none> and password file <none>
I20260929 02:20:31.747778 19137 fs_manager.cc:359] Metadata directory not provided
I20260929 02:20:31.747884 19137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:20:31.748167 19137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:20:31 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:31.798262 19137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e147dc577a674be5a811a811436e6fe3"
format_stamp: "Formatted at 2026-09-29 02:20:31 on dist-test-slave-8cs2"
server_key: "fe6595fdf0eaad59ac692e0b8a64fd44"
server_key_iv: "3fd1c3fa089be6eeef6f048864dbdbdb"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:31.799063 19137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e147dc577a674be5a811a811436e6fe3"
format_stamp: "Formatted at 2026-09-29 02:20:31 on dist-test-slave-8cs2"
server_key: "fe6595fdf0eaad59ac692e0b8a64fd44"
server_key_iv: "3fd1c3fa089be6eeef6f048864dbdbdb"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:31.804611 19137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:20:31.820391 19152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:31.821962 19137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:20:31.822122 19137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e147dc577a674be5a811a811436e6fe3"
format_stamp: "Formatted at 2026-09-29 02:20:31 on dist-test-slave-8cs2"
server_key: "fe6595fdf0eaad59ac692e0b8a64fd44"
server_key_iv: "3fd1c3fa089be6eeef6f048864dbdbdb"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:31.822340 19137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:20:31.841039 19137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:20:31.845499 19137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:20:31.845769 19137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:20:31.864131 19137 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42769
I20260929 02:20:31.864152 19203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42769 every 8 connection(s)
I20260929 02:20:31.865478 19137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:20:31.865954 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 19137
I20260929 02:20:31.866134 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:20:31.866927 7848 external_mini_cluster.cc:1428] Setting key d44fbfd7dac0877386430421a04ed76e
I20260929 02:20:31.870802 19204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:20:31 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:20:31.883796 19204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: Bootstrap starting.
I20260929 02:20:31.887229 19204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: Neither blocks nor log segments found. Creating new log.
I20260929 02:20:31.888489 19204 log.cc:824] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:20:31.889561 19207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:31.869063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.7.170.62:42769)
0929 02:20:31.869780 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:31.869795 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:31.869843 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:20:31.872070 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:31.872097 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:31.872592 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:31.873042 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:31.874077 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:31.876083 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:31.877310 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:31.878302 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:31.878482 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:31.882530 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:31.882538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:31.882553 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:31.882589 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:31.885373 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:31.886182 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:31.886188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:31.886194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:31.886344 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:31.886740 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:31.886744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:31.886746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:31.887273 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:31.887539 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:31.887970 (+ 431us) server_negotiation.cc:299] Negotiation successful
0929 02:20:31.888300 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":302,"threads_started":1}
I20260929 02:20:31.891520 19204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: No bootstrap required, opened a new log
I20260929 02:20:31.899148 19204 raft_consensus.cc:348] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } }
I20260929 02:20:31.899475 19204 raft_consensus.cc:374] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:20:31.899542 19204 raft_consensus.cc:729] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e147dc577a674be5a811a811436e6fe3, State: Initialized, Role: FOLLOWER
I20260929 02:20:31.900012 19204 consensus_queue.cc:260] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } }
I20260929 02:20:31.900183 19204 raft_consensus.cc:388] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:20:31.900273 19204 raft_consensus.cc:482] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:20:31.900388 19204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:20:31.902592 19204 raft_consensus.cc:504] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } }
I20260929 02:20:31.903021 19204 leader_election.cc:302] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e147dc577a674be5a811a811436e6fe3; no voters:
I20260929 02:20:31.903507 19204 leader_election.cc:288] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:20:31.903625 19209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:20:31.904332 19209 raft_consensus.cc:686] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [term 1 LEADER]: Becoming Leader. State: Replica: e147dc577a674be5a811a811436e6fe3, State: Running, Role: LEADER
I20260929 02:20:31.904836 19209 consensus_queue.cc:237] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } }
I20260929 02:20:31.905555 19204 sys_catalog.cc:564] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:20:31.907546 19210 sys_catalog.cc:455] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e147dc577a674be5a811a811436e6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } } }
I20260929 02:20:31.907521 19211 sys_catalog.cc:455] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e147dc577a674be5a811a811436e6fe3. Latest consensus state: current_term: 1 leader_uuid: "e147dc577a674be5a811a811436e6fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e147dc577a674be5a811a811436e6fe3" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42769 } } }
I20260929 02:20:31.908401 19210 sys_catalog.cc:458] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:20:31.908425 19211 sys_catalog.cc:458] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:20:31.912822 19215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:20:31.918002 19215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:20:31.918751 19204 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 02:20:31.933655 19215 catalog_manager.cc:1347] Generated new cluster ID: fd944d039b9747d8905177f47dd3c79f
I20260929 02:20:31.933787 19215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:20:31.951292 19215 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:20:31.953398 19215 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:20:31.963864 19215 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: Generated new TSK 0
I20260929 02:20:31.965027 19215 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:20:32.668412 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:20:32 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790648432, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 02:20:33.136395 19249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:20:33.136804 19249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:20:33.136999 19249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:20:33.148283 19249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:20:33.148491 19249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:20:33.148768 19249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:20:33.164149 19249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.170.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:20:33.165956 19249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:20:33.167685 19249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:20:33.185020 19259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:33.185020 19260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:20:33.190687 19249 server_base.cc:1034] running on GCE node
W20260929 02:20:33.191318 19262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:20:33.192128 19249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:20:33.193485 19249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:20:33.194777 19249 hybrid_clock.cc:648] HybridClock initialized: now 1790648433194710 us; error 51 us; skew 500 ppm
I20260929 02:20:33.195185 19249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:20:33 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648433, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:20:33.199481 19249 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:20:33.201481 19249 webserver.cc:460] Webserver started at http://127.7.170.1:39367/ using document root <none> and password file <none>
I20260929 02:20:33.202389 19249 fs_manager.cc:359] Metadata directory not provided
I20260929 02:20:33.202528 19249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:20:33.202905 19249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:20:33 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648433, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:33.227715 19249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "fc6fdc2a91f64e739f7b69de275bf48e"
format_stamp: "Formatted at 2026-09-29 02:20:33 on dist-test-slave-8cs2"
server_key: "3c9cdbdbd031201be5bff650e75dd7dd"
server_key_iv: "3719bd850f3b8242041771eb4322c243"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:33.228860 19249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "fc6fdc2a91f64e739f7b69de275bf48e"
format_stamp: "Formatted at 2026-09-29 02:20:33 on dist-test-slave-8cs2"
server_key: "3c9cdbdbd031201be5bff650e75dd7dd"
server_key_iv: "3719bd850f3b8242041771eb4322c243"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:33.247172 19249 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.012s sys 0.004s
I20260929 02:20:33.276609 19269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:33.278548 19249 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.006s sys 0.001s
I20260929 02:20:33.278724 19249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "fc6fdc2a91f64e739f7b69de275bf48e"
format_stamp: "Formatted at 2026-09-29 02:20:33 on dist-test-slave-8cs2"
server_key: "3c9cdbdbd031201be5bff650e75dd7dd"
server_key_iv: "3719bd850f3b8242041771eb4322c243"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:33.278980 19249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:20:33.292459 19249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:20:33.298249 19249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:20:33.298619 19249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:20:33.307650 19249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:20:33.309918 19249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:20:33.310020 19249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:33.310199 19249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:20:33.310267 19249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:33.407200 19249 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:36417
I20260929 02:20:33.407229 19381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:36417 every 8 connection(s)
I20260929 02:20:33.408969 19249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:20:33.414558 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 19249
I20260929 02:20:33.414777 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:20:33.415585 7848 external_mini_cluster.cc:1428] Setting key 16b6f1f1fa1b0a31cf95dc7acd77fdf7
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Sep 29 02:20:33 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648433, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
I20260929 02:20:33.483304 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:33.412831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:37801 (local address 127.7.170.62:42769)
0929 02:20:33.428142 (+ 15311us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:33.428151 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:33.437913 (+ 9762us) server_negotiation.cc:407] Connection header received
0929 02:20:33.445202 (+ 7289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:33.445209 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:33.445306 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:33.445397 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:33.450921 (+ 5524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:33.452394 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:33.456601 (+ 4207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:33.457215 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:33.457310 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:33.465076 (+ 7766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:33.465084 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:33.465089 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:33.465132 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:33.468788 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:33.470092 (+ 1304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:33.470099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:33.470103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:33.470354 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:33.481038 (+ 10684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:33.481046 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:33.481050 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:33.481507 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:33.481681 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:33.482675 (+ 994us) server_negotiation.cc:299] Negotiation successful
0929 02:20:33.482925 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15147,"thread_start_us":14985,"threads_started":1}
I20260929 02:20:33.484613 19385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:33.413645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42769 (local address 127.7.170.1:37801)
0929 02:20:33.437692 (+ 24047us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:33.437767 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:33.444807 (+ 7040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:33.445983 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:33.445996 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:33.446663 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:20:33.447605 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:33.447632 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:33.452789 (+ 5157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:33.452795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:33.456435 (+ 3640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:33.456450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:33.457721 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:33.457729 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:33.458016 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:33.461647 (+ 3631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:20:33.461687 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:20:33.464353 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:20:33.469052 (+ 4699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:33.469062 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:33.469081 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:33.469940 (+ 859us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:33.470532 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:33.470538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:33.470542 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:33.470819 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:33.481678 (+ 10859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:20:33.481688 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:20:33.482229 (+ 541us) client_negotiation.cc:769] Sending connection context
0929 02:20:33.482580 (+ 351us) client_negotiation.cc:240] Negotiation successful
0929 02:20:33.483191 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23648,"thread_start_us":27191,"threads_started":1}
I20260929 02:20:33.486982 19382 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42769
I20260929 02:20:33.487298 19382 heartbeater.cc:464] Registering TS with master...
I20260929 02:20:33.488166 19382 heartbeater.cc:511] Master 127.7.170.62:42769 requested a full tablet report, sending...
Sep 29 02:20:33 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648432, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:33.490828 19169 ts_manager.cc:194] Registered new tserver with Master: fc6fdc2a91f64e739f7b69de275bf48e (127.7.170.1:36417)
I20260929 02:20:33.492691 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:37801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:20:33.514341 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:20:33 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648432, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
W20260929 02:20:34.004266 19388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:20:34.004750 19388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:20:34.004968 19388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:20:34.016462 19388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:20:34.016682 19388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:20:34.016913 19388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:20:34.032469 19388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.170.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:20:34.034660 19388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:20:34.036495 19388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:20:34.072412 19395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:34.073992 19396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:34.081897 19399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:20:34.082369 19388 server_base.cc:1034] running on GCE node
I20260929 02:20:34.083621 19388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:20:34.089958 19388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:20:34.091707 19388 hybrid_clock.cc:648] HybridClock initialized: now 1790648434091607 us; error 92 us; skew 500 ppm
I20260929 02:20:34.092161 19388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:20:34 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:20:34.096385 19388 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:20:34.133394 19388 webserver.cc:460] Webserver started at http://127.7.170.2:35597/ using document root <none> and password file <none>
I20260929 02:20:34.134580 19388 fs_manager.cc:359] Metadata directory not provided
I20260929 02:20:34.134806 19388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:20:34.135200 19388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:20:34 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:34.163646 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "bcbd467e437949888f72898630f06983"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "33f0e48163730be976c97df8ccd8ef75"
server_key_iv: "84b962d6d1cad42f4b479ba2e73299d3"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.164721 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "bcbd467e437949888f72898630f06983"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "33f0e48163730be976c97df8ccd8ef75"
server_key_iv: "84b962d6d1cad42f4b479ba2e73299d3"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.171952 19388 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260929 02:20:34.201615 19412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:34.203819 19388 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.004s sys 0.012s
I20260929 02:20:34.204107 19388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "bcbd467e437949888f72898630f06983"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "33f0e48163730be976c97df8ccd8ef75"
server_key_iv: "84b962d6d1cad42f4b479ba2e73299d3"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.205099 19388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:20:34.256130 19388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:20:34.261870 19388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:20:34.262177 19388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:20:34.263340 19388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:20:34.265038 19388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:20:34.265125 19388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:34.265236 19388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:20:34.265293 19388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:34.303879 19388 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44213
I20260929 02:20:34.303918 19524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44213 every 8 connection(s)
I20260929 02:20:34.305547 19388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:20:34.308601 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 19388
I20260929 02:20:34.308765 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:20:34.309279 7848 external_mini_cluster.cc:1428] Setting key 19daceab495921c35ce357d2e6f2c55f
Sep 29 02:20:34 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
I20260929 02:20:34.344141 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:34.308807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54553 (local address 127.7.170.62:42769)
0929 02:20:34.324094 (+ 15287us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:34.324101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:34.324822 (+ 721us) server_negotiation.cc:407] Connection header received
0929 02:20:34.327317 (+ 2495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:34.327324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:34.327412 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:34.327528 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:34.329574 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:34.330703 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:34.333005 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:34.333598 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:34.333718 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:34.337690 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:34.337697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:34.337702 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:34.337741 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:34.340856 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:34.341717 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:34.341721 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:34.341723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:34.341900 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:34.342338 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:34.342343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:34.342346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:34.342687 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:34.342864 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:34.343546 (+ 682us) server_negotiation.cc:299] Negotiation successful
0929 02:20:34.343766 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15146,"thread_start_us":15292,"threads_started":1}
I20260929 02:20:34.345189 19528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:34.309570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42769 (local address 127.7.170.2:54553)
0929 02:20:34.324600 (+ 15030us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:34.324661 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:34.327008 (+ 2347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:34.327899 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:34.327913 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:34.328498 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:20:34.329374 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:34.329399 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:34.330883 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:34.330888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:34.332817 (+ 1929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:34.332840 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:34.333825 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:34.333834 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:34.334088 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:34.334821 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:20:34.334849 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:20:34.337461 (+ 2612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:20:34.341004 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:34.341013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:34.341030 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:34.341585 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:34.342006 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:34.342010 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:34.342014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:34.342233 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:34.342821 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:20:34.342829 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:20:34.343171 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 02:20:34.343429 (+ 258us) client_negotiation.cc:240] Negotiation successful
0929 02:20:34.343931 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14728,"thread_start_us":14505,"threads_started":1}
I20260929 02:20:34.346616 19525 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42769
I20260929 02:20:34.347012 19525 heartbeater.cc:464] Registering TS with master...
I20260929 02:20:34.347699 19525 heartbeater.cc:511] Master 127.7.170.62:42769 requested a full tablet report, sending...
I20260929 02:20:34.349081 19169 ts_manager.cc:194] Registered new tserver with Master: bcbd467e437949888f72898630f06983 (127.7.170.2:44213)
I20260929 02:20:34.349849 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:54553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:20:34.376643 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:52539/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:33033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:20:34.496620 19382 heartbeater.cc:503] Master 127.7.170.62:42769 was elected leader, sending a full tablet report...
W20260929 02:20:34.879408 19531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:20:34.879758 19531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:20:34.879948 19531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:20:34.890424 19531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:20:34.890606 19531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:20:34.890792 19531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:20:34.904557 19531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:33033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.170.20:52539/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:20:34.906095 19531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:20:34.907567 19531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:20:34.925767 19536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:34.926561 19537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:20:34.926730 19539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:20:34.930411 19531 server_base.cc:1034] running on GCE node
I20260929 02:20:34.930987 19531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:20:34.932106 19531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:20:34.933349 19531 hybrid_clock.cc:648] HybridClock initialized: now 1790648434933252 us; error 39 us; skew 500 ppm
I20260929 02:20:34.933789 19531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:20:34 dist-test-slave-8cs2 krb5kdc[18693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:20:34.938226 19531 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:20:34.940582 19531 webserver.cc:460] Webserver started at http://127.7.170.3:34345/ using document root <none> and password file <none>
I20260929 02:20:34.941574 19531 fs_manager.cc:359] Metadata directory not provided
I20260929 02:20:34.941752 19531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:20:34.942134 19531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:20:34 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:20:34.958797 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "373397679c4847f68ea354c7218fa6b7"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "43d424010fe12dd6c258bfa1fcf0c496"
server_key_iv: "37b47b2ffbe94227edf5039f1c35d923"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.959791 19531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "373397679c4847f68ea354c7218fa6b7"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "43d424010fe12dd6c258bfa1fcf0c496"
server_key_iv: "37b47b2ffbe94227edf5039f1c35d923"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.967119 19531 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260929 02:20:34.986240 19546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:34.987982 19531 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260929 02:20:34.988189 19531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "373397679c4847f68ea354c7218fa6b7"
format_stamp: "Formatted at 2026-09-29 02:20:34 on dist-test-slave-8cs2"
server_key: "43d424010fe12dd6c258bfa1fcf0c496"
server_key_iv: "37b47b2ffbe94227edf5039f1c35d923"
server_key_version: "kuduclusterkey@0"
I20260929 02:20:34.988469 19531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:20:35.005035 19531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:20:35.010212 19531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:20:35.010728 19531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:20:35.012382 19531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:20:35.014648 19531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:20:35.014775 19531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:35.014966 19531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:20:35.015089 19531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:20:35.101413 19531 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33159
I20260929 02:20:35.101509 19658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33159 every 8 connection(s)
I20260929 02:20:35.103127 19531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:20:35.105671 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 19531
I20260929 02:20:35.105887 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:20:35.106501 7848 external_mini_cluster.cc:1428] Setting key 69fe0e2b25cb07fce872958bd6daeebc
Sep 29 02:20:35 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648434, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:20:35.132606 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.107142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:38343 (local address 127.7.170.62:42769)
0929 02:20:35.108016 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.108022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.109913 (+ 1891us) server_negotiation.cc:407] Connection header received
0929 02:20:35.113114 (+ 3201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.113120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.113275 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.113370 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:35.115510 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.116587 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.118928 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.119652 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.119767 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.125074 (+ 5307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:35.125079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:35.125083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:35.125117 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:35.129340 (+ 4223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.130271 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.130275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.130278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.130422 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.130933 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.130936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.130938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.131255 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:35.131360 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.132109 (+ 749us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.132302 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":554,"threads_started":1}
I20260929 02:20:35.133980 19662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.108030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42769 (local address 127.7.170.3:38343)
0929 02:20:35.109655 (+ 1625us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.109701 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.112676 (+ 2975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.113725 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.113734 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.114400 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:20:35.115326 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.115355 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.116797 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.116801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.118728 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.118765 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.120003 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.120013 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.120410 (+ 397us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.121416 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:20:35.121455 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:20:35.124801 (+ 3346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:20:35.129487 (+ 4686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:35.129496 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:35.129512 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:35.130163 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:35.130526 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:20:35.130529 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:20:35.130532 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:20:35.130816 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:20:35.131352 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:20:35.131358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:20:35.131728 (+ 370us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.132019 (+ 291us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.132506 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1299,"thread_start_us":987,"threads_started":1}
I20260929 02:20:35.135756 19659 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42769
I20260929 02:20:35.136142 19659 heartbeater.cc:464] Registering TS with master...
I20260929 02:20:35.137315 19659 heartbeater.cc:511] Master 127.7.170.62:42769 requested a full tablet report, sending...
I20260929 02:20:35.139130 19169 ts_manager.cc:194] Registered new tserver with Master: 373397679c4847f68ea354c7218fa6b7 (127.7.170.3:33159)
I20260929 02:20:35.140228 19169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:38343
I20260929 02:20:35.148782 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:20:35.171547 19661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.154514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49454 (local address 127.7.170.62:42769)
0929 02:20:35.154695 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.154701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.155577 (+ 876us) server_negotiation.cc:407] Connection header received
0929 02:20:35.155732 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.155736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.155841 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.155939 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:35.158161 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.159166 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.161100 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.161787 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.161885 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.165145 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:35.165151 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:35.165154 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:35.165184 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:35.168592 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.169522 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.169526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.169529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.169708 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.170059 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.170062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.170064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.170401 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:35.170641 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.170901 (+ 260us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.171134 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:20:35.176493 19169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49454:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:20:35.188740 19169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:20:35.223169 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.206242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.7.170.1:36417)
0929 02:20:35.214030 (+ 7788us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.214039 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.214073 (+ 34us) server_negotiation.cc:407] Connection header received
0929 02:20:35.214183 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.214192 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.214492 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.214809 (+ 317us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:20:35.216226 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.218171 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.220631 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.221937 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.222056 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.222317 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.222571 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.222751 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7591,"thread_start_us":7430,"threads_started":1}
I20260929 02:20:35.223568 19672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.205630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:36417 (local address 127.0.0.1:45856)
0929 02:20:35.207015 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.207033 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.207271 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.214823 (+ 7552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.214830 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.214887 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:20:35.215972 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.215986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.218341 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.218352 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.220447 (+ 2095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.220460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.222155 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.222161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.222358 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.222404 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.222511 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.222698 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":51,"spinlock_wait_cycles":9088,"thread_start_us":759,"threads_started":1}
I20260929 02:20:35.224179 19674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.205156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44213 (local address 127.0.0.1:50500)
0929 02:20:35.206992 (+ 1836us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.207034 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.207270 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.215165 (+ 7895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.215170 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.215200 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:20:35.215972 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.215986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.218295 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.218301 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.221035 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.221048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.223223 (+ 2175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.223234 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.223480 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.223516 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.223602 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.223731 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"mutex_wait_us":78,"spinlock_wait_cycles":9856,"thread_start_us":518,"threads_started":1}
I20260929 02:20:35.224635 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.205274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.7.170.2:44213)
0929 02:20:35.214573 (+ 9299us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.214580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.214601 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:20:35.214670 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.214675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.214907 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.216031 (+ 1124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:20:35.216509 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.218168 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.221209 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.222774 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.222960 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.223148 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.223933 (+ 785us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.224097 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9112,"thread_start_us":8992,"threads_started":1}
I20260929 02:20:35.224680 19676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.206627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33159 (local address 127.0.0.1:42678)
0929 02:20:35.214221 (+ 7594us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.214239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.214459 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.215164 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.215168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.215206 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:20:35.216287 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.216300 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.217865 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.217870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.220678 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.220691 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.223512 (+ 2821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.223519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.223885 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.223917 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.224015 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.224133 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7471,"mutex_wait_us":46,"spinlock_wait_cycles":3328,"thread_start_us":7316,"threads_started":1}
I20260929 02:20:35.225545 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.207249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.7.170.3:33159)
0929 02:20:35.213655 (+ 6406us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.213665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.214519 (+ 854us) server_negotiation.cc:407] Connection header received
0929 02:20:35.214637 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.214646 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.214885 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.215153 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:20:35.216508 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.217713 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.220849 (+ 3136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.222493 (+ 1644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.222611 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.222844 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.225041 (+ 2197us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.225192 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6173,"thread_start_us":5961,"threads_started":1}
I20260929 02:20:35.228196 19317 tablet_service.cc:1467] Processing CreateTablet for tablet cf65945b2a144d13baa81e7258a016c0 (DEFAULT_TABLE table=test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:20:35.228412 19460 tablet_service.cc:1467] Processing CreateTablet for tablet cf65945b2a144d13baa81e7258a016c0 (DEFAULT_TABLE table=test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:20:35.229794 19460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf65945b2a144d13baa81e7258a016c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:20:35.229954 19317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf65945b2a144d13baa81e7258a016c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:20:35.235934 19594 tablet_service.cc:1467] Processing CreateTablet for tablet cf65945b2a144d13baa81e7258a016c0 (DEFAULT_TABLE table=test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:20:35.237321 19594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf65945b2a144d13baa81e7258a016c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:20:35.246470 19678 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Bootstrap starting.
I20260929 02:20:35.248698 19679 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Bootstrap starting.
I20260929 02:20:35.251005 19680 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Bootstrap starting.
I20260929 02:20:35.251245 19678 tablet_bootstrap.cc:654] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Neither blocks nor log segments found. Creating new log.
I20260929 02:20:35.252677 19678 log.cc:824] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Log is configured to *not* fsync() on all Append() calls
I20260929 02:20:35.252843 19679 tablet_bootstrap.cc:654] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Neither blocks nor log segments found. Creating new log.
I20260929 02:20:35.254419 19679 log.cc:824] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Log is configured to *not* fsync() on all Append() calls
I20260929 02:20:35.255280 19680 tablet_bootstrap.cc:654] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Neither blocks nor log segments found. Creating new log.
I20260929 02:20:35.256330 19678 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: No bootstrap required, opened a new log
I20260929 02:20:35.256675 19678 ts_tablet_manager.cc:1397] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.009s
I20260929 02:20:35.256920 19680 log.cc:824] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:20:35.259709 19679 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: No bootstrap required, opened a new log
I20260929 02:20:35.259970 19679 ts_tablet_manager.cc:1397] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260929 02:20:35.260219 19680 tablet_bootstrap.cc:492] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: No bootstrap required, opened a new log
I20260929 02:20:35.260506 19680 ts_tablet_manager.cc:1397] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20260929 02:20:35.268226 19679 raft_consensus.cc:348] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.268217 19678 raft_consensus.cc:348] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.268702 19678 raft_consensus.cc:374] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:20:35.268697 19679 raft_consensus.cc:374] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:20:35.268805 19678 raft_consensus.cc:729] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcbd467e437949888f72898630f06983, State: Initialized, Role: FOLLOWER
I20260929 02:20:35.268821 19679 raft_consensus.cc:729] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6fdc2a91f64e739f7b69de275bf48e, State: Initialized, Role: FOLLOWER
I20260929 02:20:35.269536 19679 consensus_queue.cc:260] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.269519 19678 consensus_queue.cc:260] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.271487 19679 ts_tablet_manager.cc:1428] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260929 02:20:35.272591 19680 raft_consensus.cc:348] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.273126 19680 raft_consensus.cc:374] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:20:35.273252 19680 raft_consensus.cc:729] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 373397679c4847f68ea354c7218fa6b7, State: Initialized, Role: FOLLOWER
I20260929 02:20:35.273409 19678 ts_tablet_manager.cc:1428] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260929 02:20:35.273603 19525 heartbeater.cc:503] Master 127.7.170.62:42769 was elected leader, sending a full tablet report...
I20260929 02:20:35.273902 19680 consensus_queue.cc:260] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.275151 19659 heartbeater.cc:503] Master 127.7.170.62:42769 was elected leader, sending a full tablet report...
I20260929 02:20:35.275753 19680 ts_tablet_manager.cc:1428] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
W20260929 02:20:35.308338 19526 tablet.cc:2369] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:20:35.336743 19684 raft_consensus.cc:482] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:20:35.337095 19684 raft_consensus.cc:504] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.338900 19684 leader_election.cc:288] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bcbd467e437949888f72898630f06983 (127.7.170.2:44213), 373397679c4847f68ea354c7218fa6b7 (127.7.170.3:33159)
I20260929 02:20:35.346886 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.339336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:40225 (local address 127.7.170.2:44213)
0929 02:20:35.339576 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.339583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.340198 (+ 615us) server_negotiation.cc:407] Connection header received
0929 02:20:35.340434 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.340439 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.340522 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.340585 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:20:35.341690 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.342616 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.344732 (+ 2116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.345824 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.345933 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.346002 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.346478 (+ 476us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.346599 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:20:35.347050 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.340497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:39239 (local address 127.7.170.3:33159)
0929 02:20:35.340712 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.340717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.341032 (+ 315us) server_negotiation.cc:407] Connection header received
0929 02:20:35.341190 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.341195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.341269 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.341362 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:20:35.342149 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.343014 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.345122 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.346107 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.346230 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.346333 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.346657 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.346783 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:20:35.347092 19688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.340177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33159 (local address 127.7.170.1:39239)
0929 02:20:35.340881 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.340898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.341074 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.341377 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.341380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.341398 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:20:35.341983 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.341996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.343174 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.343178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.344995 (+ 1817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.345010 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.346340 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.346345 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.346501 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.346530 (+ 29us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.346612 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.346728 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":403,"threads_started":1}
I20260929 02:20:35.347347 19687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.339286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44213 (local address 127.7.170.1:40225)
0929 02:20:35.340055 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0929 02:20:35.340071 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:20:35.340258 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:20:35.340800 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:20:35.340804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:20:35.340826 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:20:35.341541 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.341554 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.342740 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.342746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.344573 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:20:35.344585 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.346103 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.346110 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:20:35.346305 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.346342 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:20:35.346423 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:20:35.346518 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":56,"thread_start_us":460,"threads_started":1}
I20260929 02:20:35.348242 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf65945b2a144d13baa81e7258a016c0" candidate_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373397679c4847f68ea354c7218fa6b7" is_pre_election: true
I20260929 02:20:35.348318 19480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf65945b2a144d13baa81e7258a016c0" candidate_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbd467e437949888f72898630f06983" is_pre_election: true
I20260929 02:20:35.348783 19614 raft_consensus.cc:2454] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6fdc2a91f64e739f7b69de275bf48e in term 0.
I20260929 02:20:35.348829 19480 raft_consensus.cc:2454] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc6fdc2a91f64e739f7b69de275bf48e in term 0.
I20260929 02:20:35.349579 19272 leader_election.cc:302] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bcbd467e437949888f72898630f06983, fc6fdc2a91f64e739f7b69de275bf48e; no voters:
I20260929 02:20:35.349958 19684 raft_consensus.cc:2781] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:20:35.350116 19684 raft_consensus.cc:482] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:20:35.350232 19684 raft_consensus.cc:3037] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:20:35.352691 19684 raft_consensus.cc:504] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.353317 19684 leader_election.cc:288] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [CANDIDATE]: Term 1 election: Requested vote from peers bcbd467e437949888f72898630f06983 (127.7.170.2:44213), 373397679c4847f68ea354c7218fa6b7 (127.7.170.3:33159)
I20260929 02:20:35.353829 19614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf65945b2a144d13baa81e7258a016c0" candidate_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "373397679c4847f68ea354c7218fa6b7"
I20260929 02:20:35.353842 19480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf65945b2a144d13baa81e7258a016c0" candidate_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bcbd467e437949888f72898630f06983"
I20260929 02:20:35.354020 19614 raft_consensus.cc:3037] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:20:35.354040 19480 raft_consensus.cc:3037] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:20:35.356084 19660 tablet.cc:2369] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:20:35.356561 19614 raft_consensus.cc:2454] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6fdc2a91f64e739f7b69de275bf48e in term 1.
I20260929 02:20:35.356781 19480 raft_consensus.cc:2454] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc6fdc2a91f64e739f7b69de275bf48e in term 1.
I20260929 02:20:35.356949 19272 leader_election.cc:302] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 373397679c4847f68ea354c7218fa6b7, fc6fdc2a91f64e739f7b69de275bf48e; no voters:
I20260929 02:20:35.357280 19684 raft_consensus.cc:2781] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:20:35.357836 19684 raft_consensus.cc:686] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 1 LEADER]: Becoming Leader. State: Replica: fc6fdc2a91f64e739f7b69de275bf48e, State: Running, Role: LEADER
I20260929 02:20:35.358377 19684 consensus_queue.cc:237] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } }
I20260929 02:20:35.363226 19168 catalog_manager.cc:5495] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e reported cstate change: term changed from 0 to 1, leader changed from <none> to fc6fdc2a91f64e739f7b69de275bf48e (127.7.170.1). New cstate: current_term: 1 leader_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc6fdc2a91f64e739f7b69de275bf48e" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36417 } health_report { overall_health: HEALTHY } } }
I20260929 02:20:35.408567 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.402498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45866 (local address 127.7.170.1:36417)
0929 02:20:35.402704 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.402709 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.402729 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:20:35.402865 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.402869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.402969 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.403108 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:20:35.403537 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.404382 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.406194 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.406757 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.406882 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.407451 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:20:35.407984 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:20:35.408123 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.408216 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.408297 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260929 02:20:35.417665 19614 raft_consensus.cc:1261] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 1 FOLLOWER]: Refusing update from remote peer fc6fdc2a91f64e739f7b69de275bf48e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:20:35.417943 19480 raft_consensus.cc:1261] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 1 FOLLOWER]: Refusing update from remote peer fc6fdc2a91f64e739f7b69de275bf48e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:20:35.418713 19689 consensus_queue.cc:1035] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [LEADER]: Connected to new peer: Peer: permanent_uuid: "373397679c4847f68ea354c7218fa6b7" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:20:35.419128 19684 consensus_queue.cc:1035] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [LEADER]: Connected to new peer: Peer: permanent_uuid: "bcbd467e437949888f72898630f06983" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:20:35.446398 19168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:20:35.446686 19168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49454:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:20:35.450469 19168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e147dc577a674be5a811a811436e6fe3: Sending DeleteTablet for 3 replicas of tablet cf65945b2a144d13baa81e7258a016c0
I20260929 02:20:35.451712 19460 tablet_service.cc:1514] Processing DeleteTablet for tablet cf65945b2a144d13baa81e7258a016c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:20:35 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:50500
I20260929 02:20:35.451896 19317 tablet_service.cc:1514] Processing DeleteTablet for tablet cf65945b2a144d13baa81e7258a016c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:20:35 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45856
I20260929 02:20:35.452917 19594 tablet_service.cc:1514] Processing DeleteTablet for tablet cf65945b2a144d13baa81e7258a016c0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:20:35 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:42678
I20260929 02:20:35.454349 19704 tablet_replica.cc:330] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: stopping tablet replica
I20260929 02:20:35.454751 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 19249
I20260929 02:20:35.454779 19704 raft_consensus.cc:2229] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:20:35.455312 19704 raft_consensus.cc:2258] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:20:35.457751 19704 ts_tablet_manager.cc:1905] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:20:35.462181 19702 tablet_replica.cc:330] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: stopping tablet replica
I20260929 02:20:35.463034 19702 raft_consensus.cc:2229] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:20:35.463701 19702 raft_consensus.cc:2258] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:20:35.465525 19704 ts_tablet_manager.cc:1918] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:20:35.465709 19704 log.cc:1196] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/cf65945b2a144d13baa81e7258a016c0
I20260929 02:20:35.466302 19704 ts_tablet_manager.cc:1939] T cf65945b2a144d13baa81e7258a016c0 P 373397679c4847f68ea354c7218fa6b7: Deleting consensus metadata
I20260929 02:20:35.467815 19702 ts_tablet_manager.cc:1905] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:20:35.468544 19155 catalog_manager.cc:4841] TS 373397679c4847f68ea354c7218fa6b7 (127.7.170.3:33159): tablet cf65945b2a144d13baa81e7258a016c0 (table test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]) successfully deleted
I20260929 02:20:35.474505 19703 tablet_replica.cc:330] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: stopping tablet replica
I20260929 02:20:35.474993 19703 raft_consensus.cc:2229] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:20:35.475867 19703 raft_consensus.cc:2258] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 29 02:20:35 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:20:35.477859 19702 ts_tablet_manager.cc:1918] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:20:35.478047 19702 log.cc:1196] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/cf65945b2a144d13baa81e7258a016c0
I20260929 02:20:35.478506 19703 ts_tablet_manager.cc:1905] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:20:35.478631 19702 ts_tablet_manager.cc:1939] T cf65945b2a144d13baa81e7258a016c0 P bcbd467e437949888f72898630f06983: Deleting consensus metadata
I20260929 02:20:35.480520 19155 catalog_manager.cc:4841] TS bcbd467e437949888f72898630f06983 (127.7.170.2:44213): tablet cf65945b2a144d13baa81e7258a016c0 (table test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]) successfully deleted
I20260929 02:20:35.487169 19703 ts_tablet_manager.cc:1918] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:20:35.487383 19703 log.cc:1196] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/cf65945b2a144d13baa81e7258a016c0
I20260929 02:20:35.487675 19675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.456302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45868 (local address 127.7.170.1:36417)
0929 02:20:35.456438 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.456443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.456554 (+ 111us) server_negotiation.cc:407] Connection header received
0929 02:20:35.456804 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.456810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.456888 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.456994 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:35.469042 (+ 12048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.470414 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.472604 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.473543 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.473670 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.479363 (+ 5693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:35.479376 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:35.479394 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:35.479440 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:35.483857 (+ 4417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.484837 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.484846 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.484850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.485050 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.485521 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.485525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.485528 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.486004 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:35.486303 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.486864 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.487093 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260929 02:20:35.488030 19703 ts_tablet_manager.cc:1939] T cf65945b2a144d13baa81e7258a016c0 P fc6fdc2a91f64e739f7b69de275bf48e: Deleting consensus metadata
I20260929 02:20:35.489217 19377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:20:35.489959 19156 catalog_manager.cc:4841] TS fc6fdc2a91f64e739f7b69de275bf48e (127.7.170.1:36417): tablet cf65945b2a144d13baa81e7258a016c0 (table test-table [id=70ec4ac37b004a3e83bf0f5bf5a548ad]) successfully deleted
I20260929 02:20:35.593547 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 19249
I20260929 02:20:35.609093 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 19388
Sep 29 02:20:35 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:20:35.623080 19673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.609835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50506 (local address 127.7.170.2:44213)
0929 02:20:35.610000 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.610024 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.610043 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:20:35.610130 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.610134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.610208 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.610292 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:35.611572 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.612375 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.613688 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.614062 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.614154 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.618035 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:35.618050 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:35.618066 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:35.618101 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:35.620658 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.621348 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.621355 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.621357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.621486 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.621767 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.621770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.621771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.622146 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:35.622351 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.622651 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.622803 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:20:35.623749 19520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:20:35.713207 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 19388
I20260929 02:20:35.726349 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 19531
Sep 29 02:20:35 dist-test-slave-8cs2 krb5kdc[18693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:20:35.740357 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:20:35.727051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.7.170.3:33159)
0929 02:20:35.727228 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:20:35.727235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:20:35.727311 (+ 76us) server_negotiation.cc:407] Connection header received
0929 02:20:35.727570 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:20:35.727575 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:20:35.727655 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:20:35.727738 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:20:35.728897 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.729649 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.730880 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:20:35.731253 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:20:35.731350 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:20:35.735397 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:20:35.735412 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:20:35.735429 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:20:35.735461 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:20:35.738041 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.738731 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.738737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.738739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.738853 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:20:35.739186 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:20:35.739188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:20:35.739190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:20:35.739537 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:20:35.739747 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:20:35.739953 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:20:35.740113 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:20:35.741039 19654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:20:35.826529 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 19531
I20260929 02:20:35.839316 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 19137
I20260929 02:20:35.840394 19199 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:20:35.949191 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 19137
I20260929 02:20:35.966172 7848 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 02:20:36.300590 7848 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 02:20:36.300791 7848 mini_ranger.cc:66] Stopping Ranger...
I20260929 02:20:36.635342 7848 mini_ranger.cc:68] Stopped Ranger
2026-09-29 02:20:36.635 UTC [18723] LOG: received smart shutdown request
2026-09-29 02:20:36.638 UTC [18723] LOG: background worker "logical replication launcher" (PID 18732) exited with exit code 1
2026-09-29 02:20:36.639 UTC [18727] LOG: shutting down
2026-09-29 02:20:36.777 UTC [18723] LOG: database system is shut down
2026-09-29T02:20:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45732 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:20:36 dist-test-slave-8cs2 krb5kdc[19718](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:20:36 dist-test-slave-8cs2 krb5kdc[19718](info): set up 2 sockets
Sep 29 02:20:36 dist-test-slave-8cs2 krb5kdc[19718](info): commencing operation
krb5kdc: starting...
W20260929 02:20:37.889226 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.969s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:20:38 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:20:38Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangeradmin_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangeradmin_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangerlookup_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangerlookup_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
I20260929 02:20:38.183194 7848 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskoo3C6V/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/postgres -l logfile start
2026-09-29 02:20:40.294 UTC [19748] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-29 02:20:40.294 UTC [19748] LOG: listening on IPv4 address "127.7.170.20", port 38175
2026-09-29 02:20:40.297 UTC [19748] LOG: listening on Unix socket "/tmp/.s.PGSQL.38175"
2026-09-29 02:20:40.317 UTC [19751] LOG: database system was shut down at 2026-09-29 02:20:39 UTC
2026-09-29 02:20:40.320 UTC [19748] LOG: database system is ready to accept connections
I20260929 02:20:41.204988 7848 mini_postgres.cc:96] Postgres bound to 38175
2026-09-29 02:20:41.225 UTC [19759] FATAL: database "slave" does not exist
127.7.170.20:38175 - accepting connections
I20260929 02:20:41.226291 7848 mini_ranger.cc:161] Starting Ranger...
I20260929 02:20:41.249754 7848 mini_ranger.cc:84] Created miniranger Postgres user
I20260929 02:20:41.654460 7848 mini_ranger.cc:87] Created ranger Postgres database
I20260929 02:20:41.654569 7848 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin
2026-09-29 02:20:41,922 [I] DB FLAVOR :POSTGRES
2026-09-29 02:20:41,922 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:20:41,922 [I] Checking connection..
2026-09-29 02:20:41,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-29 02:20:42,148 [I] Checking connection passed.
2026-09-29 02:20:42,148 [I] --------- Verifying version history table ---------
2026-09-29 02:20:42,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:20:42,396 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:20:42,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:20:42,634 [I] Table x_db_version_h does not exist in database ranger
2026-09-29 02:20:42,634 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-29 02:20:42,634 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-29 02:20:42.864 UTC [19823] WARNING: there is no transaction in progress
2026-09-29 02:20:42,869 [I] create_dbversion_catalog.sql file imported successfully
2026-09-29 02:20:42,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-29 02:20:43,111 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-29 02:20:43,112 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:20:43,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-29 02:20:43,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-29 02:20:43,595 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-8cs2.c.gcp-upstream.internal','N') ;"
2026-09-29 02:20:43,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:20:44,061 [I] Table x_portal_user does not exist in database ranger
2026-09-29 02:20:44,061 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:20:44,290 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-29 02:20:44,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:20:44,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:20:44,773 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-29 02:20:44,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-29 02:20:45.649 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.666 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.682 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.696 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.710 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.726 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.785 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.790 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.803 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.812 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.822 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.835 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.843 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.850 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.855 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:45.861 UTC [19931] WARNING: there is no transaction in progress
2026-09-29 02:20:46,389 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-29 02:20:46,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-29 02:20:46,622 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-29 02:20:46,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-29 02:20:46,863 [I] Table x_portal_user already exists in database 'ranger'
2026-09-29 02:20:46,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-29 02:20:47,092 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-29 02:20:47,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-29 02:20:47,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-29 02:20:47,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-8cs2.c.gcp-upstream.internal';"
2026-09-29 02:20:47,823 [I] CORE_DB_SCHEMA import status has been updated
2026-09-29 02:20:47,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-29 02:20:48,058 [I] DB_PATCHES have already been applied
I20260929 02:20:48.063808 7848 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskoo3C6V/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260929 02:20:48.063890 7848 mini_ranger.cc:193] Using host: 127.7.170.20
I20260929 02:20:48.078626 7848 mini_ranger.cc:227] Ranger admin URL: http://127.7.170.20:32919
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-admin/ews/webapp].
Sep 29, 2026 2:20:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260929 02:20:49.105813 7848 mini_ranger.cc:160] Time spent starting Ranger: real 7.880s user 0.000s sys 0.041s
Sep 29, 2026 2:21:08 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 29, 2026 2:21:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 29, 2026 2:21:08 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 29, 2026 2:21:08 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 29, 2026 2:21:09 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260929 02:21:09.821982 7848 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.170.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.170.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangerkms_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.170.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/rangerkms_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.170.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.170.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.170.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/HTTP_127.7.170.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 29 02:21:09 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260929 02:21:09.974586 7848 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260929 02:21:09.994256 7848 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260929 02:21:10.691615 7848 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260929 02:21:10.691715 7848 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms
I20260929 02:21:10.691752 7848 mini_ranger_kms.cc:227] Using postgres at 127.7.170.20:38175
2026-09-29 02:21:11,019 [I] DB FLAVOR :POSTGRES
2026-09-29 02:21:11,019 [I] --------- Verifying Ranger DB connection ---------
2026-09-29 02:21:11,020 [I] Checking connection
2026-09-29 02:21:11,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-29 02:21:11,255 [I] connection success
2026-09-29 02:21:11,255 [I] --------- Verifying Ranger DB tables ---------
2026-09-29 02:21:11,255 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-29 02:21:11,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-29 02:21:11,480 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-29 02:21:11,480 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-29 02:21:11,481 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-29 02:21:11,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.170.20:38175/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-29 02:21:11,723 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260929 02:21:12.180796 7848 mini_ranger_kms.cc:310] Created kms service
I20260929 02:21:12.372258 7848 mini_ranger_kms.cc:326] Created kudu user
I20260929 02:21:12.412184 7848 mini_ranger_kms.cc:343] Created rangerkms user
Sep 29 02:21:12 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648469, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:12.796579 7848 mini_ranger_kms.cc:384] Added ranger policy
I20260929 02:21:12.796918 7848 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms:/tmp/dist-test-taskoo3C6V/build/asan/bin/postgresql.jar:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskoo3C6V/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260929 02:21:12.797019 7848 mini_ranger_kms.cc:240] Using host: 127.7.170.20
I20260929 02:21:12.811740 7848 mini_ranger_kms.cc:276] Ranger KMS PID: 20111
I20260929 02:21:12.811852 7848 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.170.20:39815
Sep 29, 2026 2:21:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 29, 2026 2:21:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 29, 2026 2:21:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 29, 2026 2:21:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 29, 2026 2:21:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-kms/ews/webapp].
Sep 29, 2026 2:21:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260929 02:21:14.842607 7848 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.868s user 0.067s sys 0.278s
I20260929 02:21:14.842832 7848 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260929 02:21:14.842980 7848 mini_ranger_kms.cc:401] 127.7.170.20:39815/kms/v1/keys
Sep 29 02:21:14 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790648474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 29 02:21:15 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
Sep 29 02:21:15 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648474, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.170.20@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
Sep 29 02:21:18 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:21:18.141054 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42763
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42763
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:18.469098 20162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:18.469373 20162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:18.469499 20162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:18.476887 20162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:18.477015 20162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:18.477090 20162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:18.477164 20162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:18.477227 20162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:18.486871 20162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.170.20:39815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42763
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:18.488023 20162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:18.489280 20162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:18.500129 20168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:18.500171 20170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:18.500209 20167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:18.500574 20162 server_base.cc:1034] running on GCE node
I20260929 02:21:18.501356 20162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:18.502592 20162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:18.503814 20162 hybrid_clock.cc:648] HybridClock initialized: now 1790648478503755 us; error 42 us; skew 500 ppm
I20260929 02:21:18.504189 20162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:18 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:18.516590 20162 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:18.518527 20162 webserver.cc:460] Webserver started at http://127.7.170.62:42141/ using document root <none> and password file <none>
I20260929 02:21:18.519434 20162 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:18.519603 20162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:18.520035 20162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:21:18 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:18.592257 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "b0164bb12d9843f6bcdd953f82755460"
format_stamp: "Formatted at 2026-09-29 02:21:18 on dist-test-slave-8cs2"
server_key: "7f98923c46a1cf0947d52db20133bb85"
server_key_iv: "0b0aed6ccccc41b693c609762f7e57ce"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:18.593488 20162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "b0164bb12d9843f6bcdd953f82755460"
format_stamp: "Formatted at 2026-09-29 02:21:18 on dist-test-slave-8cs2"
server_key: "7f98923c46a1cf0947d52db20133bb85"
server_key_iv: "0b0aed6ccccc41b693c609762f7e57ce"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:18.600505 20162 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260929 02:21:18.621165 20177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:18.622936 20162 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:21:18.623135 20162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "b0164bb12d9843f6bcdd953f82755460"
format_stamp: "Formatted at 2026-09-29 02:21:18 on dist-test-slave-8cs2"
server_key: "7f98923c46a1cf0947d52db20133bb85"
server_key_iv: "0b0aed6ccccc41b693c609762f7e57ce"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:18.623409 20162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:18.662420 20162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:18.667146 20162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:18.667476 20162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:18.685428 20162 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42763
I20260929 02:21:18.685406 20228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42763 every 8 connection(s)
I20260929 02:21:18.687207 20162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:18.689069 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20162
I20260929 02:21:18.689302 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:21:18.690121 7848 external_mini_cluster.cc:1428] Setting key 55b2b8166c8be5236dff07982b1991af
I20260929 02:21:18.692063 20229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:18.709057 20229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Bootstrap starting.
I20260929 02:21:18.712968 20229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:18.714248 20229 log.cc:824] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:18.717021 20229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: No bootstrap required, opened a new log
Sep 29 02:21:18 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:18.727192 20229 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:18.727633 20229 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:18.727737 20229 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0164bb12d9843f6bcdd953f82755460, State: Initialized, Role: FOLLOWER
I20260929 02:21:18.727898 20232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:18.692236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.7.170.62:42763)
0929 02:21:18.705601 (+ 13365us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:18.705622 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:18.706674 (+ 1052us) server_negotiation.cc:407] Connection header received
0929 02:21:18.708553 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:18.708578 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:18.709033 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:18.709438 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:18.710777 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:18.712066 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:18.713425 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:18.714120 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:18.714284 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:18.718879 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:18.718889 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:18.718908 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:18.718955 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:18.722827 (+ 3872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:18.723658 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:18.723666 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:18.723676 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:18.723880 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:18.724286 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:18.724290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:18.724293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:18.725088 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:18.725463 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:18.726012 (+ 549us) server_negotiation.cc:299] Negotiation successful
0929 02:21:18.726579 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13029,"thread_start_us":12783,"threads_started":1}
I20260929 02:21:18.728387 20229 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:18.728653 20229 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:18.728778 20229 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:18.729017 20229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:18.731591 20229 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:18.732195 20229 leader_election.cc:302] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0164bb12d9843f6bcdd953f82755460; no voters:
I20260929 02:21:18.737715 20229 leader_election.cc:288] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:21:18.737803 20234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:18.744490 20234 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 LEADER]: Becoming Leader. State: Replica: b0164bb12d9843f6bcdd953f82755460, State: Running, Role: LEADER
I20260929 02:21:18.745076 20234 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:18.746392 20229 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:18.748263 20236 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0164bb12d9843f6bcdd953f82755460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } } }
I20260929 02:21:18.748554 20236 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:18.749267 20235 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0164bb12d9843f6bcdd953f82755460. Latest consensus state: current_term: 1 leader_uuid: "b0164bb12d9843f6bcdd953f82755460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } } }
I20260929 02:21:18.750659 20235 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:18.754202 20239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:18.764410 20239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:18.765206 20229 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260929 02:21:18.784703 20239 catalog_manager.cc:1347] Generated new cluster ID: 9de4d1e8403342dfb0a58ae169cde4a2
I20260929 02:21:18.784817 20239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:18.802759 20239 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:21:18.805301 20239 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:18.817327 20239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Generated new TSK 0
I20260929 02:21:18.818496 20239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:21:19.528389 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:19 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790648479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260929 02:21:19.954663 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:19.955180 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:19.955392 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:19.967317 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:19.967483 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:19.967736 20274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:19.984103 20274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:19.985885 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:19.987566 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:20.015764 20284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:20.018021 20285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:20.019552 20287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:20.020642 20274 server_base.cc:1034] running on GCE node
I20260929 02:21:20.021510 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:20.022826 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:20.024145 20274 hybrid_clock.cc:648] HybridClock initialized: now 1790648480024068 us; error 64 us; skew 500 ppm
I20260929 02:21:20.024569 20274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:20.030054 20274 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:20.036609 20274 webserver.cc:460] Webserver started at http://127.7.170.1:39431/ using document root <none> and password file <none>
I20260929 02:21:20.037525 20274 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:20.037649 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:20.037954 20274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:20.056595 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "dce17873b427296d7754368dc6fd41df"
server_key_iv: "33fdcb24bd75c5a955d36d461fb872d7"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.057842 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "dce17873b427296d7754368dc6fd41df"
server_key_iv: "33fdcb24bd75c5a955d36d461fb872d7"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.065559 20274 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260929 02:21:20.103292 20294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.105254 20274 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.010s sys 0.013s
I20260929 02:21:20.105515 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "dce17873b427296d7754368dc6fd41df"
server_key_iv: "33fdcb24bd75c5a955d36d461fb872d7"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.105872 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:20.132547 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:20.137202 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:20.137506 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:20.138571 20274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:20.140085 20274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:20.140169 20274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.140316 20274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:20.140370 20274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.203652 20274 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:44661
I20260929 02:21:20.203696 20406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:44661 every 8 connection(s)
I20260929 02:21:20.205495 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:20.214926 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20274
I20260929 02:21:20.215170 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:21:20.215945 7848 external_mini_cluster.cc:1428] Setting key f6cb52599e0d03475d7e1ca7ecd76bf5
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:20.270994 20410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:20.209979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:36281 (local address 127.7.170.62:42763)
0929 02:21:20.242386 (+ 32407us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:20.242393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:20.245032 (+ 2639us) server_negotiation.cc:407] Connection header received
0929 02:21:20.249745 (+ 4713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:20.249753 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:20.249870 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:20.249955 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:20.253785 (+ 3830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.254905 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.257554 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.258233 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.258347 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:20.262928 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:20.262935 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:20.262940 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:20.262986 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:20.266702 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:20.267790 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:20.267796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:20.267800 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:20.267978 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:20.268605 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:20.268610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:20.268614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:20.269009 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:20.269160 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:20.270359 (+ 1199us) server_negotiation.cc:299] Negotiation successful
0929 02:21:20.270608 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32247,"thread_start_us":32419,"threads_started":1}
I20260929 02:21:20.273397 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:20.209223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.1:36281)
0929 02:21:20.243174 (+ 33951us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:20.243237 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:20.248300 (+ 5063us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:20.250594 (+ 2294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:20.250608 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:20.251343 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:20.252217 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:20.252251 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.255135 (+ 2884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.255140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:20.257346 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:20.257364 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.258747 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.258756 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:20.259070 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:20.259915 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:20.259947 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:20.262603 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:20.266933 (+ 4330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:20.266943 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:20.266964 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:20.267623 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:20.268159 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:20.268165 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:20.268169 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:20.268468 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:20.269428 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:20.269438 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:20.269898 (+ 460us) client_negotiation.cc:769] Sending connection context
0929 02:21:20.271309 (+ 1411us) client_negotiation.cc:240] Negotiation successful
0929 02:21:20.271898 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33557,"thread_start_us":39355,"threads_started":1}
I20260929 02:21:20.275674 20407 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:20.276010 20407 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:20.276891 20407 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:20.279824 20194 ts_manager.cc:194] Registered new tserver with Master: 6af1712f1fb24cff910dc8c7ed4f1fa5 (127.7.170.1:44661)
I20260929 02:21:20.281872 20194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:36281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:21:20.293394 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648479, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
W20260929 02:21:20.755846 20413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:20.756333 20413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:20.756561 20413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:20.767633 20413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:20.767838 20413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:20.768049 20413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:20.783761 20413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:20.785658 20413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:20.787380 20413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:20.815023 20420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:20.816164 20421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:20.824735 20423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:20.825812 20413 server_base.cc:1034] running on GCE node
I20260929 02:21:20.826606 20413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:20.827862 20413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:20.831283 20413 hybrid_clock.cc:648] HybridClock initialized: now 1790648480831166 us; error 105 us; skew 500 ppm
I20260929 02:21:20.831683 20413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:20.835944 20413 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:20.844694 20413 webserver.cc:460] Webserver started at http://127.7.170.2:42025/ using document root <none> and password file <none>
I20260929 02:21:20.845897 20413 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:20.846138 20413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:20.846580 20413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:20.869480 20413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "5ec3cf817c594439b80cd3da032900d6"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "145bdb3af24435b7d92378c827df47bc"
server_key_iv: "eb39535499747e63730af42fead35c31"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.870342 20413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "5ec3cf817c594439b80cd3da032900d6"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "145bdb3af24435b7d92378c827df47bc"
server_key_iv: "eb39535499747e63730af42fead35c31"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.875873 20413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:21:20.888417 20437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.890044 20413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:21:20.890213 20413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "5ec3cf817c594439b80cd3da032900d6"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "145bdb3af24435b7d92378c827df47bc"
server_key_iv: "eb39535499747e63730af42fead35c31"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:20.890429 20413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:20.911664 20413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:20.916065 20413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:20.916356 20413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:20.917371 20413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:20.918768 20413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:20.918853 20413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.918972 20413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:20.919020 20413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:20.953030 20413 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34257
I20260929 02:21:20.953032 20549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34257 every 8 connection(s)
I20260929 02:21:20.954396 20413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:20.964294 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20413
I20260929 02:21:20.964483 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:21:20.965116 7848 external_mini_cluster.cc:1428] Setting key 3e71f110d86e1f9df30952e20df56d96
Sep 29 02:21:20 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:20.977216 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:20.957562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:34847 (local address 127.7.170.62:42763)
0929 02:21:20.958034 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:20.958041 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:20.959113 (+ 1072us) server_negotiation.cc:407] Connection header received
0929 02:21:20.960950 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:20.960957 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:20.961065 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:20.961202 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:20.962627 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.963746 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.965514 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.966075 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.966192 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:20.970103 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:20.970109 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:20.970114 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:20.970154 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:20.973571 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:20.974447 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:20.974453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:20.974457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:20.974618 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:20.975092 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:20.975097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:20.975100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:20.975485 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:20.975648 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:20.976536 (+ 888us) server_negotiation.cc:299] Negotiation successful
0929 02:21:20.976783 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":213,"threads_started":1}
I20260929 02:21:20.978258 20553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:20.958137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.2:34847)
0929 02:21:20.958947 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:20.958993 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:20.960649 (+ 1656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:20.961397 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:20.961406 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:20.961843 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:20.962443 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:20.962462 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.963917 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.963921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:20.965372 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:20.965385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:20.966297 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:20.966304 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:20.966554 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:20.967191 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:20.967222 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:20.969845 (+ 2623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:20.973734 (+ 3889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:20.973744 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:20.973762 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:20.974317 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:20.974731 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:20.974736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:20.974739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:20.974969 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:20.975635 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:20.975643 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:20.976099 (+ 456us) client_negotiation.cc:769] Sending connection context
0929 02:21:20.976416 (+ 317us) client_negotiation.cc:240] Negotiation successful
0929 02:21:20.976842 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":321,"threads_started":1}
I20260929 02:21:20.979988 20550 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:20.980361 20550 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:20.981333 20550 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:20.983188 20194 ts_manager.cc:194] Registered new tserver with Master: 5ec3cf817c594439b80cd3da032900d6 (127.7.170.2:34257)
I20260929 02:21:20.984258 20194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:34847
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:21:21.020005 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:21.285691 20407 heartbeater.cc:503] Master 127.7.170.62:42763 was elected leader, sending a full tablet report...
W20260929 02:21:21.336967 20556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:21.337275 20556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:21.337400 20556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:21.344643 20556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:21.344789 20556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:21.344961 20556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:21.355093 20556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:21.356443 20556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:21.357750 20556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:21.370661 20561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:21.370615 20562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:21.370636 20564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:21.371678 20556 server_base.cc:1034] running on GCE node
I20260929 02:21:21.372303 20556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:21.373405 20556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:21.374619 20556 hybrid_clock.cc:648] HybridClock initialized: now 1790648481374570 us; error 40 us; skew 500 ppm
I20260929 02:21:21.374975 20556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:21.378093 20556 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:21.379537 20556 webserver.cc:460] Webserver started at http://127.7.170.3:39121/ using document root <none> and password file <none>
I20260929 02:21:21.380244 20556 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:21.380358 20556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:21.380663 20556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:21.396001 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "614128e9a95447c5864d6f8daa1be612"
format_stamp: "Formatted at 2026-09-29 02:21:21 on dist-test-slave-8cs2"
server_key: "022c7b19d664bb88db2cd33494282984"
server_key_iv: "0487716f51d63f37e0459f0be696069a"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:21.396920 20556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "614128e9a95447c5864d6f8daa1be612"
format_stamp: "Formatted at 2026-09-29 02:21:21 on dist-test-slave-8cs2"
server_key: "022c7b19d664bb88db2cd33494282984"
server_key_iv: "0487716f51d63f37e0459f0be696069a"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:21.402329 20556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:21:21.420153 20571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:21.421751 20556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:21:21.421923 20556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "614128e9a95447c5864d6f8daa1be612"
format_stamp: "Formatted at 2026-09-29 02:21:21 on dist-test-slave-8cs2"
server_key: "022c7b19d664bb88db2cd33494282984"
server_key_iv: "0487716f51d63f37e0459f0be696069a"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:21.422149 20556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:21.443917 20556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:21.448083 20556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:21.448380 20556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:21.449388 20556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:21.450753 20556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:21.450829 20556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:21.450924 20556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:21.450982 20556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:21.482164 20556 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36289
I20260929 02:21:21.482205 20683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36289 every 8 connection(s)
I20260929 02:21:21.483451 20556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:21:21.492203 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20556
I20260929 02:21:21.492386 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:21:21.492838 7848 external_mini_cluster.cc:1428] Setting key 28065133fc4e91a2f106f91ebe0203ae
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648481, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:21.502774 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:21.486043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:43547 (local address 127.7.170.62:42763)
0929 02:21:21.486430 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:21.486434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:21.487466 (+ 1032us) server_negotiation.cc:407] Connection header received
0929 02:21:21.489292 (+ 1826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:21.489298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:21.489377 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:21.489471 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:21.490898 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.491625 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.493238 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.493818 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.493957 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:21.497215 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:21.497222 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:21.497225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:21.497263 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:21.499610 (+ 2347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.500282 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.500288 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.500290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.500410 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.500795 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.500799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.500801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.501104 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:21.501254 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:21.502182 (+ 928us) server_negotiation.cc:299] Negotiation successful
0929 02:21:21.502429 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":186,"threads_started":1}
I20260929 02:21:21.503723 20687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:21.486623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.3:43547)
0929 02:21:21.487319 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:21.487358 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:21.489016 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:21.489719 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:21.489728 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:21.490149 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:21.490728 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:21.490745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.491768 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.491771 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:21.493109 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:21.493120 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.494025 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.494032 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:21.494276 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:21.494940 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:21.494969 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:21.497017 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:21.499733 (+ 2716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:21.499743 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:21.499761 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:21.500166 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:21.500510 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:21.500515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:21.500518 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:21.500688 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:21.501445 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:21.501459 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:21.501768 (+ 309us) client_negotiation.cc:769] Sending connection context
0929 02:21:21.502076 (+ 308us) client_negotiation.cc:240] Negotiation successful
0929 02:21:21.502593 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":284,"threads_started":1}
I20260929 02:21:21.505084 20684 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:21.505362 20684 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:21.506093 20684 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:21.507707 20194 ts_manager.cc:194] Registered new tserver with Master: 614128e9a95447c5864d6f8daa1be612 (127.7.170.3:36289)
I20260929 02:21:21.508520 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:21:21.508631 20194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:43547
I20260929 02:21:21.510756 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20274
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:21:21.527586 20689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:21.511698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60730 (local address 127.7.170.1:44661)
0929 02:21:21.512292 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:21.512301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:21.512329 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:21:21.512401 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:21.512409 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:21.512611 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:21.512757 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:21.514254 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.515272 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.516806 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.517441 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.517562 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:21.521663 (+ 4101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:21.521677 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:21.521692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:21.521730 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:21.524509 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.525350 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.525357 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.525360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.525534 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.525947 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.525951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.525954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.526346 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:21.526651 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:21.526969 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 02:21:21.527234 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":264,"threads_started":1}
I20260929 02:21:21.529014 20402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:21.608528 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20274
I20260929 02:21:21.621150 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20413
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:21:21.637298 20691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:21.621996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.7.170.2:34257)
0929 02:21:21.622848 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:21.622863 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:21.622888 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:21:21.622945 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:21.622952 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:21.623252 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:21.623429 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:21.624989 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.626147 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.627635 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.628191 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.628310 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:21.632351 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:21.632361 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:21.632375 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:21.632408 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:21.634835 (+ 2427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.635442 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.635448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.635450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.635570 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.635853 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.635856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.635857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.636189 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:21.636409 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:21.636683 (+ 274us) server_negotiation.cc:299] Negotiation successful
0929 02:21:21.636913 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":705,"thread_start_us":574,"threads_started":1}
I20260929 02:21:21.638526 20545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:21.717923 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20413
I20260929 02:21:21.731958 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20556
Sep 29 02:21:21 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:21:21.747316 20693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:21.732843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38600 (local address 127.7.170.3:36289)
0929 02:21:21.733489 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:21.733496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:21.733524 (+ 28us) server_negotiation.cc:407] Connection header received
0929 02:21:21.733579 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:21.733585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:21.733796 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:21.733945 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:21.735292 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.736283 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.737609 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:21.738167 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:21.738279 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:21.742350 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:21.742361 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:21.742374 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:21.742406 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:21.744903 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.745542 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.745548 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.745551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.745669 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:21.745970 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:21.745973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:21.745975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:21.746260 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:21.746501 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:21.746837 (+ 336us) server_negotiation.cc:299] Negotiation successful
0929 02:21:21.747042 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":340,"threads_started":1}
I20260929 02:21:21.748499 20679 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:21.828311 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20556
I20260929 02:21:21.840801 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20162
I20260929 02:21:21.842042 20224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:21.945374 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20162
I20260929 02:21:21.967767 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42763
--webserver_interface=127.7.170.62
--webserver_port=42141
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42763
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:22.278198 20696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:22.278476 20696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:22.278723 20696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:22.285897 20696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:22.285998 20696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:22.286070 20696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:22.286149 20696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:22.286213 20696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:22.295913 20696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42763
--ranger_config_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=42141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:22.297170 20696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:22.298509 20696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:22.310328 20701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:22.310381 20704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:22.310288 20702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:22.311594 20696 server_base.cc:1034] running on GCE node
I20260929 02:21:22.312506 20696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:22.314220 20696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:22.315454 20696 hybrid_clock.cc:648] HybridClock initialized: now 1790648482315402 us; error 37 us; skew 500 ppm
I20260929 02:21:22.315862 20696 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:22 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:22.319239 20696 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:22.320976 20696 webserver.cc:460] Webserver started at http://127.7.170.62:42141/ using document root <none> and password file <none>
I20260929 02:21:22.321715 20696 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:22.321841 20696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:21:22 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:22.397296 20696 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:21:22.400813 20712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:22.402333 20696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:22.402531 20696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "b0164bb12d9843f6bcdd953f82755460"
format_stamp: "Formatted at 2026-09-29 02:21:18 on dist-test-slave-8cs2"
server_key: "7f98923c46a1cf0947d52db20133bb85"
server_key_iv: "0b0aed6ccccc41b693c609762f7e57ce"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:22.403553 20696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:22.422672 20696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:22.426843 20696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:22.427131 20696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:22.445622 20696 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42763
I20260929 02:21:22.445642 20763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42763 every 8 connection(s)
I20260929 02:21:22.447256 20696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:22.449333 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20696
I20260929 02:21:22.450385 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:44661
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:42763
--webserver_port=39431
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:22.456801 20764 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:21:22.460968 20764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Bootstrap starting.
I20260929 02:21:22.479923 20764 log.cc:824] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:22.488606 20764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:21:22.489126 20764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Bootstrap complete.
I20260929 02:21:22.497583 20764 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:22.498433 20764 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0164bb12d9843f6bcdd953f82755460, State: Initialized, Role: FOLLOWER
I20260929 02:21:22.499027 20764 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:22.499220 20764 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:22.499326 20764 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:22.499485 20764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:21:22.503232 20764 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:22.503705 20764 leader_election.cc:302] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0164bb12d9843f6bcdd953f82755460; no voters:
I20260929 02:21:22.504349 20764 leader_election.cc:288] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:21:22.504521 20768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:21:22.505533 20768 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [term 2 LEADER]: Becoming Leader. State: Replica: b0164bb12d9843f6bcdd953f82755460, State: Running, Role: LEADER
I20260929 02:21:22.506135 20768 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } }
I20260929 02:21:22.506574 20764 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:22.508605 20770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0164bb12d9843f6bcdd953f82755460. Latest consensus state: current_term: 2 leader_uuid: "b0164bb12d9843f6bcdd953f82755460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } } }
I20260929 02:21:22.508971 20770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:22.509509 20769 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b0164bb12d9843f6bcdd953f82755460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0164bb12d9843f6bcdd953f82755460" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42763 } } }
I20260929 02:21:22.509773 20769 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:22.520218 20774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:22.525653 20774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:22.527127 20774 catalog_manager.cc:1259] Loaded cluster ID: 9de4d1e8403342dfb0a58ae169cde4a2
I20260929 02:21:22.527276 20774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:22.531167 20774 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:22.532577 20774 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Loaded TSK: 0
I20260929 02:21:22.534135 20774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:21:22.538033 20764 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260929 02:21:22.902204 20766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:22.902737 20766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:22.902940 20766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:22.914073 20766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:22.914286 20766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:22.914537 20766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:22.930040 20766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:44661
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=39431
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:22.932114 20766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:22.933916 20766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:22.952299 20801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:22.953805 20802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:22.961282 20804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:22.962834 20766 server_base.cc:1034] running on GCE node
I20260929 02:21:22.963562 20766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:22.964852 20766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:22.966236 20766 hybrid_clock.cc:648] HybridClock initialized: now 1790648482966171 us; error 56 us; skew 500 ppm
I20260929 02:21:22.966620 20766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:22 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:22.970618 20766 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:22.978277 20766 webserver.cc:460] Webserver started at http://127.7.170.1:39431/ using document root <none> and password file <none>
I20260929 02:21:22.979007 20766 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:22.979137 20766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:21:22 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:23.020088 20766 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:21:23.023681 20812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.025298 20766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:23.025488 20766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "dce17873b427296d7754368dc6fd41df"
server_key_iv: "33fdcb24bd75c5a955d36d461fb872d7"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:23.026569 20766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:23.051162 20766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:23.057921 20766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:23.058319 20766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:23.059475 20766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:23.063509 20766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:23.063606 20766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.063714 20766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:23.063786 20766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.119674 20766 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:44661
I20260929 02:21:23.119727 20925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:44661 every 8 connection(s)
I20260929 02:21:23.121279 20766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:23.122786 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20766
I20260929 02:21:23.123399 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:34257
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:42763
--webserver_port=42025
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:23 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:23.156340 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:23.130858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.1:54443)
0929 02:21:23.133831 (+ 2973us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:23.133879 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:23.135736 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:23.137391 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:23.137402 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:23.138000 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:23.138532 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:23.138553 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.140864 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.140869 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:23.143126 (+ 2257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:23.143161 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.144954 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.144962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:23.145226 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:23.145911 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:23.145945 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:23.148592 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:23.152173 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:23.152182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:23.152198 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:23.152593 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:23.153024 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:23.153029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:23.153033 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:23.153201 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:23.153950 (+ 749us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:23.153957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:23.154258 (+ 301us) client_negotiation.cc:769] Sending connection context
0929 02:21:23.154497 (+ 239us) client_negotiation.cc:240] Negotiation successful
0929 02:21:23.155023 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2687,"thread_start_us":2484,"threads_started":1}
I20260929 02:21:23.156325 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:23.130840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:54443 (local address 127.7.170.62:42763)
0929 02:21:23.131635 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:23.131653 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:23.134011 (+ 2358us) server_negotiation.cc:407] Connection header received
0929 02:21:23.136292 (+ 2281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:23.136313 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:23.136793 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:23.137238 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:23.138737 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.140654 (+ 1917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.143335 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.144358 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.144520 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:23.148829 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:23.148836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:23.148850 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:23.148889 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:23.152012 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:23.152736 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:23.152744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:23.152752 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:23.152908 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:23.153310 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:23.153315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:23.153319 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:23.153805 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:23.153977 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:23.154743 (+ 766us) server_negotiation.cc:299] Negotiation successful
0929 02:21:23.155123 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":350,"threads_started":1}
I20260929 02:21:23.160054 20926 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:23.160351 20926 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:23.161106 20926 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
W20260929 02:21:23.164289 20926 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:42763 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:21:23.164515 20926 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:23.164774 20926 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:23.165882 20926 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:23.166204 20926 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
W20260929 02:21:23.167184 20926 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:21:23.528471 20928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:23.528761 20928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:23.528954 20928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:23.536748 20928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:23.536895 20928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:23.537101 20928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:23.547545 20928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:34257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=42025
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:23.548832 20928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:23.550172 20928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:23.566715 20945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:23.566741 20944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:23.567391 20947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:23.568305 20928 server_base.cc:1034] running on GCE node
I20260929 02:21:23.569271 20928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:23.570358 20928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:23.571578 20928 hybrid_clock.cc:648] HybridClock initialized: now 1790648483571498 us; error 75 us; skew 500 ppm
I20260929 02:21:23.571882 20928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:23 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:23.580699 20928 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:23.605311 20928 webserver.cc:460] Webserver started at http://127.7.170.2:42025/ using document root <none> and password file <none>
I20260929 02:21:23.606590 20928 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:23.606775 20928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:21:23 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:23.656385 20928 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.005s sys 0.008s
I20260929 02:21:23.660274 20957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.662182 20928 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260929 02:21:23.662372 20928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "5ec3cf817c594439b80cd3da032900d6"
format_stamp: "Formatted at 2026-09-29 02:21:20 on dist-test-slave-8cs2"
server_key: "145bdb3af24435b7d92378c827df47bc"
server_key_iv: "eb39535499747e63730af42fead35c31"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:23.663782 20928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Sep 29 02:21:23 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:23.687346 20928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:23.692731 20928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:23.693099 20928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:23.700515 20928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:23.709334 20928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:23.709461 20928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.709575 20928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:23.709651 20928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:23.773911 20928 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34257
I20260929 02:21:23.773949 21070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34257 every 8 connection(s)
I20260929 02:21:23.775559 20928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:23.777117 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 20928
I20260929 02:21:23.777848 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.170.20:39815/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:36289
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:42763
--webserver_port=39121
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:23 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:23.827485 21072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:23.785101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.2:58835)
0929 02:21:23.789723 (+ 4622us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:23.789778 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:23.792153 (+ 2375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:23.806579 (+ 14426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:23.806595 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:23.807367 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:23.808346 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:23.808375 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.811223 (+ 2848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.811229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:23.813391 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:23.813409 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.814558 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.814566 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:23.814870 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:23.815670 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:23.815702 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:23.818492 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:23.822629 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:23.822639 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:23.822659 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:23.823285 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:23.823749 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:23.823755 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:23.823759 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:23.824030 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:23.824706 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:23.824715 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:23.825188 (+ 473us) client_negotiation.cc:769] Sending connection context
0929 02:21:23.825478 (+ 290us) client_negotiation.cc:240] Negotiation successful
0929 02:21:23.826082 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4285,"thread_start_us":4060,"threads_started":1}
I20260929 02:21:23.828394 21073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:23.785760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:58835 (local address 127.7.170.62:42763)
0929 02:21:23.805652 (+ 19892us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:23.805660 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:23.805691 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:21:23.805768 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:23.805775 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:23.805888 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:23.808622 (+ 2734us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:23.808811 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.810012 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.813613 (+ 3601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:23.814300 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:23.815316 (+ 1016us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:23.818788 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:23.818795 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:23.818801 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:23.818843 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:23.822452 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:23.823459 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:23.823465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:23.823469 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:23.823621 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:23.824186 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:23.824192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:23.824195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:23.824560 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:23.827630 (+ 3070us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:23.827798 (+ 168us) server_negotiation.cc:299] Negotiation successful
0929 02:21:23.828047 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19727,"thread_start_us":20283,"threads_started":1}
I20260929 02:21:23.829885 21071 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:23.830222 21071 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:23.831120 21071 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:23.833289 20729 ts_manager.cc:194] Registered new tserver with Master: 5ec3cf817c594439b80cd3da032900d6 (127.7.170.2:34257)
I20260929 02:21:23.835356 20729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:58835
Sep 29 02:21:24 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:24.167348 20926 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:24.167798 20926 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:24.169227 20729 ts_manager.cc:194] Registered new tserver with Master: 6af1712f1fb24cff910dc8c7ed4f1fa5 (127.7.170.1:44661)
I20260929 02:21:24.170400 20729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:54443
W20260929 02:21:24.222208 21074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:24.222550 21074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:24.222692 21074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:24.230033 21074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:24.230154 21074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:24.230321 21074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:24.243214 21074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.170.20:39815/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:36289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=39121
--tserver_master_addrs=127.7.170.62:42763
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:24.244915 21074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:24.246649 21074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:24.262615 21080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:24.263124 21081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:24.271322 21083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:24.271807 21074 server_base.cc:1034] running on GCE node
I20260929 02:21:24.272535 21074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:24.273759 21074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:24.274982 21074 hybrid_clock.cc:648] HybridClock initialized: now 1790648484274914 us; error 56 us; skew 500 ppm
I20260929 02:21:24.275354 21074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:24 dist-test-slave-8cs2 krb5kdc[19718](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:24.279284 21074 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:24.281169 21074 webserver.cc:460] Webserver started at http://127.7.170.3:39121/ using document root <none> and password file <none>
I20260929 02:21:24.282042 21074 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:24.282179 21074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 29 02:21:24 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:24.304872 21074 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:21:24.308632 21091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:24.310424 21074 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260929 02:21:24.310582 21074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "614128e9a95447c5864d6f8daa1be612"
format_stamp: "Formatted at 2026-09-29 02:21:21 on dist-test-slave-8cs2"
server_key: "022c7b19d664bb88db2cd33494282984"
server_key_iv: "0487716f51d63f37e0459f0be696069a"
server_key_version: "kuduclusterkey@0"
I20260929 02:21:24.311805 21074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:24.340714 21074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:24.345813 21074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:24.346100 21074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:24.348112 21074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:24.356091 21074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:24.356204 21074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:24.356302 21074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:24.356374 21074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 29 02:21:24 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for HTTP/127.7.170.20@KRBTEST.COM
I20260929 02:21:24.481756 21204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36289 every 8 connection(s)
I20260929 02:21:24.482106 21074 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36289
I20260929 02:21:24.483997 21074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:21:24.490244 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21074
Sep 29 02:21:24 dist-test-slave-8cs2 krb5kdc[19718](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648484, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:24.533449 21207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.491376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43096 (local address 127.7.170.62:42763)
0929 02:21:24.507268 (+ 15892us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.507272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.507283 (+ 11us) server_negotiation.cc:407] Connection header received
0929 02:21:24.507338 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.507342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.507418 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.507527 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:24.510033 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.511083 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.513817 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.514417 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.514541 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.523261 (+ 8720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:24.523267 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:24.523272 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:24.523308 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:24.527710 (+ 4402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.528409 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.528415 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.528418 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.528555 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.529534 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.529539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.529543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.530144 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:24.530404 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.531948 (+ 1544us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.533067 (+ 1119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15812,"thread_start_us":21625,"threads_started":1}
I20260929 02:21:24.537765 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.489693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:49393 (local address 127.7.170.62:42763)
0929 02:21:24.506894 (+ 17201us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.506899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.506917 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:21:24.511200 (+ 4283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.511205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.511275 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.511367 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:24.517119 (+ 5752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.518303 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.520369 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.521086 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.521207 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.529173 (+ 7966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:24.529181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:24.529186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:24.529220 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:24.532285 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.534149 (+ 1864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.534155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.534227 (+ 72us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.534362 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.535042 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.535047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.535051 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.535388 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:24.535519 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.537023 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.537277 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17033,"thread_start_us":23225,"threads_started":1}
I20260929 02:21:24.540230 21208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.490349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42763 (local address 127.7.170.3:49393)
0929 02:21:24.502861 (+ 12512us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.502925 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.510785 (+ 7860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.513757 (+ 2972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.513769 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.514388 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:24.515471 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.515499 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.518509 (+ 3010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.518514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.520223 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.520243 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.522601 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.522607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.522902 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.523644 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:24.523672 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:24.528916 (+ 5244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:24.532438 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:24.532447 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:24.532465 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:24.534013 (+ 1548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:24.534510 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:24.534515 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:24.534519 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:24.534748 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:24.535607 (+ 859us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:24.535615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:24.536501 (+ 886us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.538276 (+ 1775us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.539046 (+ 770us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12113,"thread_start_us":14459,"threads_started":1}
I20260929 02:21:24.541986 21205 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42763
I20260929 02:21:24.542287 21205 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:24.543133 21205 heartbeater.cc:511] Master 127.7.170.62:42763 requested a full tablet report, sending...
I20260929 02:21:24.544807 20729 ts_manager.cc:194] Registered new tserver with Master: 614128e9a95447c5864d6f8daa1be612 (127.7.170.3:36289)
I20260929 02:21:24.545920 20729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:49393
I20260929 02:21:24.550578 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:21:24.583230 21210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.559636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.7.170.62:42763)
0929 02:21:24.559800 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.559807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.570328 (+ 10521us) server_negotiation.cc:407] Connection header received
0929 02:21:24.570798 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.570805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.570892 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.570982 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:24.573174 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.573883 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.575194 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.575627 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.575732 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.578194 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:24.578199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:24.578202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:24.578235 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:24.580719 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.581405 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.581409 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.581412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.581537 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.581879 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.581883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.581885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.582150 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:24.582360 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.582692 (+ 332us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.582914 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:21:24.587934 20729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:21:24.595458 20729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:21:24.647692 21224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.610771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44446)
0929 02:21:24.633823 (+ 23052us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.633873 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.634095 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.637065 (+ 2970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.637073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.637100 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:24.638935 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.638952 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641227 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.641233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.643913 (+ 2680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.644540 (+ 627us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.646639 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.646645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.646836 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.646891 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.647037 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.647228 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22720,"mutex_wait_us":45,"thread_start_us":24098,"threads_started":1}
I20260929 02:21:24.648492 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.620443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38608 (local address 127.7.170.3:36289)
0929 02:21:24.634729 (+ 14286us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.634738 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.636814 (+ 2076us) server_negotiation.cc:407] Connection header received
0929 02:21:24.636895 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.636900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.637154 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.637338 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:24.639572 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641200 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.644076 (+ 2876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.645570 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.645687 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.647909 (+ 2222us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.647982 (+ 73us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.648164 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14083,"thread_start_us":14234,"threads_started":1}
I20260929 02:21:24.648582 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.636244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.7.170.2:34257)
0929 02:21:24.636859 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.636868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.637563 (+ 695us) server_negotiation.cc:407] Connection header received
0929 02:21:24.637840 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.637847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.638795 (+ 948us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.638978 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:24.639785 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641239 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.645695 (+ 4456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.647180 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.647284 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.647466 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.648100 (+ 634us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.648273 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":304,"threads_started":1}
I20260929 02:21:24.648514 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.611336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44446 (local address 127.7.170.1:44661)
0929 02:21:24.635371 (+ 24035us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.635381 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.635411 (+ 30us) server_negotiation.cc:407] Connection header received
0929 02:21:24.635513 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.635521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.635842 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.636049 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:24.639373 (+ 3324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641036 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.644695 (+ 3659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.646164 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.646274 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.646487 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.648065 (+ 1578us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.648204 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23860,"thread_start_us":24754,"threads_started":1}
I20260929 02:21:24.647810 21220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.619821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36289 (local address 127.0.0.1:38608)
0929 02:21:24.636566 (+ 16745us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.636585 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.636701 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.637631 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.637637 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.637696 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:24.639353 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.639371 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641359 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.641364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.643915 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.643932 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.647057 (+ 3125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.647063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.647217 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.647244 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.647311 (+ 67us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.647434 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16603,"mutex_wait_us":46,"spinlock_wait_cycles":1792,"thread_start_us":13517,"threads_started":1}
I20260929 02:21:24.649375 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.634498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34257 (local address 127.0.0.1:58840)
0929 02:21:24.637463 (+ 2965us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.637481 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.637665 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.639195 (+ 1530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.639199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.639226 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:24.639606 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.639617 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.641369 (+ 1752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.641373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.645515 (+ 4142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.645531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.647704 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.647711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.647894 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.647925 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.648831 (+ 906us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.648986 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2738,"mutex_wait_us":1488,"thread_start_us":2585,"threads_started":1}
I20260929 02:21:24.653007 21005 tablet_service.cc:1467] Processing CreateTablet for tablet f56665b6fd144cae93bd9972f8555c2d (DEFAULT_TABLE table=test-table [id=b4d33a3b3222428a91a41f4dae322942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:24.653012 20861 tablet_service.cc:1467] Processing CreateTablet for tablet f56665b6fd144cae93bd9972f8555c2d (DEFAULT_TABLE table=test-table [id=b4d33a3b3222428a91a41f4dae322942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:24.654418 21005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56665b6fd144cae93bd9972f8555c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:24.654438 20861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56665b6fd144cae93bd9972f8555c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:24.655774 21140 tablet_service.cc:1467] Processing CreateTablet for tablet f56665b6fd144cae93bd9972f8555c2d (DEFAULT_TABLE table=test-table [id=b4d33a3b3222428a91a41f4dae322942]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:24.657626 21140 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f56665b6fd144cae93bd9972f8555c2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:24.674588 21235 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Bootstrap starting.
I20260929 02:21:24.679486 21233 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Bootstrap starting.
I20260929 02:21:24.681329 21234 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: Bootstrap starting.
I20260929 02:21:24.682324 21235 tablet_bootstrap.cc:654] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:24.683367 21235 log.cc:824] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:24.683812 21233 tablet_bootstrap.cc:654] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:24.685066 21233 log.cc:824] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:24.686693 21234 tablet_bootstrap.cc:654] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:24.687885 21234 log.cc:824] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:24.689644 21235 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: No bootstrap required, opened a new log
I20260929 02:21:24.689859 21233 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: No bootstrap required, opened a new log
I20260929 02:21:24.689937 21235 ts_tablet_manager.cc:1397] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.000s
I20260929 02:21:24.690241 21233 ts_tablet_manager.cc:1397] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20260929 02:21:24.700111 21235 raft_consensus.cc:348] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.700595 21235 raft_consensus.cc:374] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:24.700692 21235 raft_consensus.cc:729] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 614128e9a95447c5864d6f8daa1be612, State: Initialized, Role: FOLLOWER
I20260929 02:21:24.701469 21235 consensus_queue.cc:260] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.702806 21205 heartbeater.cc:503] Master 127.7.170.62:42763 was elected leader, sending a full tablet report...
I20260929 02:21:24.703840 21235 ts_tablet_manager.cc:1428] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260929 02:21:24.704530 21233 raft_consensus.cc:348] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.704892 21234 tablet_bootstrap.cc:492] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: No bootstrap required, opened a new log
I20260929 02:21:24.704984 21233 raft_consensus.cc:374] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:24.705087 21233 raft_consensus.cc:729] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ec3cf817c594439b80cd3da032900d6, State: Initialized, Role: FOLLOWER
I20260929 02:21:24.705241 21234 ts_tablet_manager.cc:1397] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: Time spent bootstrapping tablet: real 0.024s user 0.018s sys 0.003s
I20260929 02:21:24.705679 21233 consensus_queue.cc:260] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.712652 21071 heartbeater.cc:503] Master 127.7.170.62:42763 was elected leader, sending a full tablet report...
I20260929 02:21:24.713357 21233 ts_tablet_manager.cc:1428] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Time spent starting tablet: real 0.023s user 0.014s sys 0.006s
I20260929 02:21:24.716173 21234 raft_consensus.cc:348] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.716632 21234 raft_consensus.cc:374] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:24.716724 21234 raft_consensus.cc:729] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6af1712f1fb24cff910dc8c7ed4f1fa5, State: Initialized, Role: FOLLOWER
I20260929 02:21:24.717437 21234 consensus_queue.cc:260] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.718472 20926 heartbeater.cc:503] Master 127.7.170.62:42763 was elected leader, sending a full tablet report...
I20260929 02:21:24.719201 21234 ts_tablet_manager.cc:1428] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260929 02:21:24.760689 21206 tablet.cc:2369] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:21:24.768064 21240 raft_consensus.cc:482] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:21:24.768342 21240 raft_consensus.cc:504] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.770084 21240 leader_election.cc:288] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6af1712f1fb24cff910dc8c7ed4f1fa5 (127.7.170.1:44661), 614128e9a95447c5864d6f8daa1be612 (127.7.170.3:36289)
I20260929 02:21:24.780601 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.770657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:44393 (local address 127.7.170.1:44661)
0929 02:21:24.771070 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.771077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.772754 (+ 1677us) server_negotiation.cc:407] Connection header received
0929 02:21:24.773088 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.773094 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.773181 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.773246 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:24.774122 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.775276 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.778079 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.779337 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.779442 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.779554 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.780173 (+ 619us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.780327 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249}
I20260929 02:21:24.781427 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.770270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.7.170.2:44393)
0929 02:21:24.772657 (+ 2387us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.772681 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.772883 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.773460 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.773464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.773491 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:24.773949 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.773962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.775468 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.775474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.777875 (+ 2401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.777891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.779759 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.779766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.779987 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.780023 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.780869 (+ 846us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.781063 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2244,"thread_start_us":2063,"threads_started":1}
I20260929 02:21:24.782517 20881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56665b6fd144cae93bd9972f8555c2d" candidate_uuid: "5ec3cf817c594439b80cd3da032900d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" is_pre_election: true
I20260929 02:21:24.783092 20881 raft_consensus.cc:2454] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ec3cf817c594439b80cd3da032900d6 in term 0.
I20260929 02:21:24.783799 20960 leader_election.cc:302] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec3cf817c594439b80cd3da032900d6, 6af1712f1fb24cff910dc8c7ed4f1fa5; no voters:
I20260929 02:21:24.784221 21240 raft_consensus.cc:2781] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:21:24.784358 21240 raft_consensus.cc:482] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:21:24.784456 21240 raft_consensus.cc:3037] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:24.787679 21240 raft_consensus.cc:504] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.788358 21240 leader_election.cc:288] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [CANDIDATE]: Term 1 election: Requested vote from peers 6af1712f1fb24cff910dc8c7ed4f1fa5 (127.7.170.1:44661), 614128e9a95447c5864d6f8daa1be612 (127.7.170.3:36289)
I20260929 02:21:24.789008 20881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56665b6fd144cae93bd9972f8555c2d" candidate_uuid: "5ec3cf817c594439b80cd3da032900d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5"
I20260929 02:21:24.789135 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.773159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56461 (local address 127.7.170.3:36289)
0929 02:21:24.773391 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.773398 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.781820 (+ 8422us) server_negotiation.cc:407] Connection header received
0929 02:21:24.782106 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.782113 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.782197 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.782256 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:24.782796 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.783908 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.786553 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.787822 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.787935 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.788044 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.788791 (+ 747us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.788954 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:21:24.789216 20881 raft_consensus.cc:3037] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:24.789594 21242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.773328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36289 (local address 127.7.170.2:56461)
0929 02:21:24.781736 (+ 8408us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:24.781753 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:24.781940 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:24.782438 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:24.782444 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:24.782470 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:24.782633 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.782643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.784133 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.784138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.786351 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:24.786367 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.788298 (+ 1931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.788305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:24.788598 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.788634 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:21:24.789283 (+ 649us) client_negotiation.cc:240] Negotiation successful
0929 02:21:24.789417 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8300,"thread_start_us":15381,"threads_started":1}
I20260929 02:21:24.790580 21159 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56665b6fd144cae93bd9972f8555c2d" candidate_uuid: "5ec3cf817c594439b80cd3da032900d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "614128e9a95447c5864d6f8daa1be612"
I20260929 02:21:24.790557 21160 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f56665b6fd144cae93bd9972f8555c2d" candidate_uuid: "5ec3cf817c594439b80cd3da032900d6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "614128e9a95447c5864d6f8daa1be612" is_pre_election: true
I20260929 02:21:24.791018 21159 raft_consensus.cc:3037] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:24.792253 20881 raft_consensus.cc:2454] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec3cf817c594439b80cd3da032900d6 in term 1.
I20260929 02:21:24.792636 20960 leader_election.cc:302] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ec3cf817c594439b80cd3da032900d6, 6af1712f1fb24cff910dc8c7ed4f1fa5; no voters:
I20260929 02:21:24.792912 21240 raft_consensus.cc:2781] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:24.793458 21240 raft_consensus.cc:686] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 1 LEADER]: Becoming Leader. State: Replica: 5ec3cf817c594439b80cd3da032900d6, State: Running, Role: LEADER
I20260929 02:21:24.794081 21159 raft_consensus.cc:2454] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ec3cf817c594439b80cd3da032900d6 in term 1.
I20260929 02:21:24.793988 21240 consensus_queue.cc:237] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } }
I20260929 02:21:24.798249 20729 catalog_manager.cc:5495] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ec3cf817c594439b80cd3da032900d6 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "5ec3cf817c594439b80cd3da032900d6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ec3cf817c594439b80cd3da032900d6" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34257 } health_report { overall_health: HEALTHY } } }
I20260929 02:21:24.814108 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.808958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58850 (local address 127.7.170.2:34257)
0929 02:21:24.809151 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.809157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.809292 (+ 135us) server_negotiation.cc:407] Connection header received
0929 02:21:24.809497 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.809501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.809596 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.809715 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:21:24.810079 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.810820 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.812109 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.812493 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.812597 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.813077 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:21:24.813568 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:21:24.813699 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.813773 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.813853 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260929 02:21:24.815696 21075 tablet.cc:2369] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:21:24.821910 21159 raft_consensus.cc:1261] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 1 FOLLOWER]: Refusing update from remote peer 5ec3cf817c594439b80cd3da032900d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:24.822129 20881 raft_consensus.cc:1261] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 1 FOLLOWER]: Refusing update from remote peer 5ec3cf817c594439b80cd3da032900d6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:24.822872 21244 consensus_queue.cc:1035] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "614128e9a95447c5864d6f8daa1be612" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:24.823207 21240 consensus_queue.cc:1035] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6af1712f1fb24cff910dc8c7ed4f1fa5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 44661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:24.836864 21247 mvcc.cc:205] Tried to move back new op lower bound from 7334496193822162944 to 7334496193719898112. Current Snapshot: MvccSnapshot[applied={T|T < 7334496193822162944}]
I20260929 02:21:24.859881 20729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:24.860119 20729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:24.864259 20729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0164bb12d9843f6bcdd953f82755460: Sending DeleteTablet for 3 replicas of tablet f56665b6fd144cae93bd9972f8555c2d
I20260929 02:21:24.865322 20861 tablet_service.cc:1514] Processing DeleteTablet for tablet f56665b6fd144cae93bd9972f8555c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:24 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:44446
I20260929 02:21:24.865322 21140 tablet_service.cc:1514] Processing DeleteTablet for tablet f56665b6fd144cae93bd9972f8555c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:24 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:38608
I20260929 02:21:24.868695 21005 tablet_service.cc:1514] Processing DeleteTablet for tablet f56665b6fd144cae93bd9972f8555c2d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:24 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:58840
I20260929 02:21:24.883744 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20766
I20260929 02:21:24.904536 21221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:24.885195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44448 (local address 127.7.170.1:44661)
0929 02:21:24.885392 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:24.885398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:24.886138 (+ 740us) server_negotiation.cc:407] Connection header received
0929 02:21:24.886214 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:24.886220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:24.886318 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:24.886408 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:24.888361 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.889678 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.891582 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:24.892270 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:24.892383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:24.895953 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:24.895974 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:24.895998 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:24.896060 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:24.900196 (+ 4136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.901132 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.901143 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.901148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.901323 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:24.901764 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:24.901769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:24.901773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:24.902259 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:24.902582 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:24.903858 (+ 1276us) server_negotiation.cc:299] Negotiation successful
0929 02:21:24.904112 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 02:21:24.905292 20921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:24.924515 21258 tablet_replica.cc:330] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: stopping tablet replica
I20260929 02:21:24.925066 21258 raft_consensus.cc:2229] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:21:24.926803 21258 raft_consensus.cc:2258] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:24.927490 21257 tablet_replica.cc:330] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: stopping tablet replica
I20260929 02:21:24.927842 21257 raft_consensus.cc:2229] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:24.928252 21257 raft_consensus.cc:2258] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:24.930320 21258 ts_tablet_manager.cc:1905] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:24.936566 21257 ts_tablet_manager.cc:1905] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:24.937675 21258 ts_tablet_manager.cc:1918] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:24.937829 21258 log.cc:1196] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/f56665b6fd144cae93bd9972f8555c2d
I20260929 02:21:24.938354 21258 ts_tablet_manager.cc:1939] T f56665b6fd144cae93bd9972f8555c2d P 5ec3cf817c594439b80cd3da032900d6: Deleting consensus metadata
I20260929 02:21:24.940135 20714 catalog_manager.cc:4841] TS 5ec3cf817c594439b80cd3da032900d6 (127.7.170.2:34257): tablet f56665b6fd144cae93bd9972f8555c2d (table test-table [id=b4d33a3b3222428a91a41f4dae322942]) successfully deleted
I20260929 02:21:24.943190 21257 ts_tablet_manager.cc:1918] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:24.943334 21257 log.cc:1196] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/f56665b6fd144cae93bd9972f8555c2d
I20260929 02:21:24.943789 21257 ts_tablet_manager.cc:1939] T f56665b6fd144cae93bd9972f8555c2d P 614128e9a95447c5864d6f8daa1be612: Deleting consensus metadata
I20260929 02:21:24.945482 20714 catalog_manager.cc:4841] TS 614128e9a95447c5864d6f8daa1be612 (127.7.170.3:36289): tablet f56665b6fd144cae93bd9972f8555c2d (table test-table [id=b4d33a3b3222428a91a41f4dae322942]) successfully deleted
I20260929 02:21:25.024365 21256 tablet_replica.cc:330] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5: stopping tablet replica
I20260929 02:21:25.024778 21256 raft_consensus.cc:2229] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:25.025331 21256 raft_consensus.cc:2258] T f56665b6fd144cae93bd9972f8555c2d P 6af1712f1fb24cff910dc8c7ed4f1fa5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:25.026209 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20766
I20260929 02:21:25.039953 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20928
W20260929 02:21:25.040073 20715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.170.1:44661 (error 108)
I20260929 02:21:25.042224 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.041732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44454)
0929 02:21:25.041917 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.042047 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260929 02:21:25.042490 20715 catalog_manager.cc:4584] TS 6af1712f1fb24cff910dc8c7ed4f1fa5 (127.7.170.1:44661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f56665b6fd144cae93bd9972f8555c2d: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
I20260929 02:21:25.055635 21229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.040666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58856 (local address 127.7.170.2:34257)
0929 02:21:25.040843 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:25.040849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:25.040869 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:21:25.041171 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:25.041178 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:25.041254 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:25.041373 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:25.042947 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:25.044156 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:25.045595 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:25.046256 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:25.046380 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:25.048699 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:25.048714 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:25.048732 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:25.048770 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:25.052555 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:25.053385 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:25.053394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:25.053398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:25.053562 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:25.054002 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:25.054008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:25.054011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:25.054493 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:25.054812 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:25.055055 (+ 243us) server_negotiation.cc:299] Negotiation successful
0929 02:21:25.055262 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:21:25.056309 21066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:25.096109 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.095549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44456)
0929 02:21:25.095779 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.095900 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 02:21:25.097160 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.096757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44470)
0929 02:21:25.096893 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.097026 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260929 02:21:25.157734 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.157224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44474)
0929 02:21:25.157436 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.157548 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 02:21:25.158689 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.158342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44484)
0929 02:21:25.158476 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.158559 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 02:21:25.168993 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20928
I20260929 02:21:25.183383 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21074
I20260929 02:21:25.196192 21225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.184117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38612 (local address 127.7.170.3:36289)
0929 02:21:25.184272 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:25.184276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:25.184294 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:21:25.184417 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:25.184421 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:25.184509 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:25.184591 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:25.185853 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:25.186590 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:25.188085 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:25.188481 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:25.188599 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:25.190921 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:25.190936 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:25.190952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:25.190989 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:25.193754 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:25.194359 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:25.194367 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:25.194370 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:25.194512 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:25.194833 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:25.194838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:25.194840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:25.195244 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:25.195482 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:25.195749 (+ 267us) server_negotiation.cc:299] Negotiation successful
0929 02:21:25.195913 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:21:25.196883 21200 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:25.239303 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.238814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44492)
0929 02:21:25.239035 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.239130 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260929 02:21:25.240268 21228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:25.239899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:44661 (local address 127.0.0.1:44498)
0929 02:21:25.240089 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:25.240157 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260929 02:21:25.283002 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21074
I20260929 02:21:25.295956 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 20696
I20260929 02:21:25.297130 20759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:25.406702 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 20696
I20260929 02:21:25.424300 7848 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260929 02:21:25.759135 7848 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260929 02:21:25.759369 7848 mini_ranger.cc:66] Stopping Ranger...
I20260929 02:21:26.093743 7848 mini_ranger.cc:68] Stopped Ranger
2026-09-29 02:21:26.093 UTC [19748] LOG: received smart shutdown request
2026-09-29 02:21:26.096 UTC [19748] LOG: background worker "logical replication launcher" (PID 19757) exited with exit code 1
2026-09-29 02:21:26.096 UTC [19752] LOG: shutting down
2026-09-29 02:21:26.236 UTC [19748] LOG: database system is shut down
2026-09-29T02:21:26Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49460 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:21:26 dist-test-slave-8cs2 krb5kdc[21272](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:21:26 dist-test-slave-8cs2 krb5kdc[21272](info): set up 2 sockets
Sep 29 02:21:26 dist-test-slave-8cs2 krb5kdc[21272](info): commencing operation
krb5kdc: starting...
W20260929 02:21:27.353308 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.972s user 0.006s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:21:27 dist-test-slave-8cs2 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:21:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:21:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:21:27.570268 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:45633
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:27.884382 21285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:27.884648 21285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:27.884738 21285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:27.891691 21285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:27.891781 21285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:27.891832 21285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:27.891893 21285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:27.891934 21285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:27.901266 21285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:45633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:27.902431 21285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:27.903609 21285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:27.914500 21290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:27.914525 21291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:27.914731 21293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:27.915400 21285 server_base.cc:1034] running on GCE node
I20260929 02:21:27.916128 21285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:27.917541 21285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:27.918748 21285 hybrid_clock.cc:648] HybridClock initialized: now 1790648487918703 us; error 30 us; skew 500 ppm
I20260929 02:21:27.919092 21285 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:27 dist-test-slave-8cs2 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:27.922106 21285 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:27.923424 21285 webserver.cc:460] Webserver started at http://127.7.170.62:40247/ using document root <none> and password file <none>
I20260929 02:21:27.924069 21285 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:27.924186 21285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:27.924453 21285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:27.927340 21285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e178e35dd7a949199cedf4b0e849d91a"
format_stamp: "Formatted at 2026-09-29 02:21:27 on dist-test-slave-8cs2"
server_key: "9be5482aa24102196c920401e9a15b9f"
server_key_iv: "6cd99e84b13cd0fa3bdf1fb1ad07e693"
server_key_version: "encryptionkey@0"
I20260929 02:21:27.928089 21285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e178e35dd7a949199cedf4b0e849d91a"
format_stamp: "Formatted at 2026-09-29 02:21:27 on dist-test-slave-8cs2"
server_key: "9be5482aa24102196c920401e9a15b9f"
server_key_iv: "6cd99e84b13cd0fa3bdf1fb1ad07e693"
server_key_version: "encryptionkey@0"
I20260929 02:21:27.932822 21285 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260929 02:21:27.935993 21300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:27.937323 21285 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:21:27.937477 21285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e178e35dd7a949199cedf4b0e849d91a"
format_stamp: "Formatted at 2026-09-29 02:21:27 on dist-test-slave-8cs2"
server_key: "9be5482aa24102196c920401e9a15b9f"
server_key_iv: "6cd99e84b13cd0fa3bdf1fb1ad07e693"
server_key_version: "encryptionkey@0"
I20260929 02:21:27.937638 21285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:27.964006 21285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:27.968050 21285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:27.968298 21285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:27.983985 21285 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:45633
I20260929 02:21:27.984031 21351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:45633 every 8 connection(s)
I20260929 02:21:27.985256 21285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:27.988780 21352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:27.991307 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21285
I20260929 02:21:27.991525 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:21:27.992312 7848 external_mini_cluster.cc:1428] Setting key b1cf6200886b283346b82e2bc38b71b5
I20260929 02:21:27.996896 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.61@KRBTEST.COM" created.
I20260929 02:21:28.020776 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.000400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38400 (local address 127.7.170.62:45633)
0929 02:21:28.001914 (+ 1514us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.001920 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.001960 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:21:28.003971 (+ 2011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.003981 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.004202 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.004359 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.005881 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.007068 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.008994 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.009526 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.009651 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.013579 (+ 3928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.013587 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.013599 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.013629 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.016407 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.017131 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.017137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.017140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.017298 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.017649 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.017652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.017654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.018227 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.018417 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.018924 (+ 507us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.019301 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":1334,"spinlock_wait_cycles":16256,"thread_start_us":1136,"threads_started":1}
I20260929 02:21:28.020793 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.000399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:45633 (local address 127.0.0.1:38400)
0929 02:21:28.001285 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.001329 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.003621 (+ 2292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.004450 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.004457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.005233 (+ 776us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.005667 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.005693 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.007215 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.007220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.008835 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.008858 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.009746 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.009756 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.009915 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.010528 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.010554 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.013312 (+ 2758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.016530 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.016539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.016549 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.017013 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.017403 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.017407 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.017410 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.017563 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.018352 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.018359 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.018571 (+ 212us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.018790 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.019419 (+ 629us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"mutex_wait_us":152,"thread_start_us":404,"threads_started":1}
I20260929 02:21:28.023789 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:21:28.024876 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.024429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55260)
0929 02:21:28.024605 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.024733 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 02:21:28.025326 21302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
I20260929 02:21:28.026847 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.026546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55266)
0929 02:21:28.026660 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.026727 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:21:28.027024 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36953: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
I20260929 02:21:28.044567 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:36953
--webserver_interface=127.7.170.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:28.076345 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } attempt: 1
I20260929 02:21:28.077499 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.077062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55272)
0929 02:21:28.077249 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.077364 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 02:21:28.078387 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.078054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55280)
0929 02:21:28.078188 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.078253 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:21:28.078547 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36953: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
I20260929 02:21:28.146780 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } attempt: 2
I20260929 02:21:28.147907 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.147437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55294)
0929 02:21:28.147667 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.147768 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 02:21:28.148830 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.148487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55302)
0929 02:21:28.148626 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.148701 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 02:21:28.149027 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36953: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
I20260929 02:21:28.240274 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } attempt: 3
I20260929 02:21:28.241562 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.241086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55312)
0929 02:21:28.241268 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.241417 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 02:21:28.242499 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.242169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55322)
0929 02:21:28.242299 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.242366 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260929 02:21:28.242657 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36953: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
W20260929 02:21:28.351642 21359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:28.351917 21359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:28.352056 21359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:28.359127 21359 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:28.359234 21359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:28.359309 21359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:28.359390 21359 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:28.359447 21359 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:28.368804 21359 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:36953
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.170.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:28.370038 21359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:28.371304 21359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:28.382091 21364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:28.382063 21365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:28.382213 21367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:28.382781 21359 server_base.cc:1034] running on GCE node
I20260929 02:21:28.383495 21359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:28.384788 21359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:28.385891 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } attempt: 4
I20260929 02:21:28.385983 21359 hybrid_clock.cc:648] HybridClock initialized: now 1790648488385933 us; error 55 us; skew 500 ppm
I20260929 02:21:28.386293 21359 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:21:28.386935 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.386440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55328)
0929 02:21:28.386714 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.386797 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260929 02:21:28.387976 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.387581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55330)
0929 02:21:28.387745 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.387827 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260929 02:21:28.388155 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36953: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36953: connect: Connection refused (error 111)
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:28.389465 21359 init.cc:377] Logged in from keytab as kudu/127.7.170.61@KRBTEST.COM (short username kudu)
I20260929 02:21:28.390759 21359 webserver.cc:460] Webserver started at http://127.7.170.61:41303/ using document root <none> and password file <none>
I20260929 02:21:28.391455 21359 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:28.391566 21359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:28.391847 21359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:28.394923 21359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data/instance:
uuid: "6966aa130db646118ccf5a14a8b2d46c"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "3902d89dacbfe9fa60c56913450efd84"
server_key_iv: "f4a28ec93c7096bb478abdbc9b100378"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.395694 21359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal/instance:
uuid: "6966aa130db646118ccf5a14a8b2d46c"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "3902d89dacbfe9fa60c56913450efd84"
server_key_iv: "f4a28ec93c7096bb478abdbc9b100378"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.400596 21359 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:21:28.403858 21374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:28.405217 21359 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:21:28.405371 21359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "6966aa130db646118ccf5a14a8b2d46c"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "3902d89dacbfe9fa60c56913450efd84"
server_key_iv: "f4a28ec93c7096bb478abdbc9b100378"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.405544 21359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:28.416805 21359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:28.420697 21359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:28.420969 21359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:28.436789 21359 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.61:36953
I20260929 02:21:28.436816 21425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.61:36953 every 8 connection(s)
I20260929 02:21:28.438059 21359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
I20260929 02:21:28.441454 21426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:28.444670 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21359
I20260929 02:21:28.444851 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-1/wal/instance
I20260929 02:21:28.445294 7848 external_mini_cluster.cc:1428] Setting key 1328f2b78695c3d04aef43396f24d7ae
I20260929 02:21:28.447660 21426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.60@KRBTEST.COM" created.
I20260929 02:21:28.466174 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.450013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.7.170.62:45633)
0929 02:21:28.450208 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.450214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.451408 (+ 1194us) server_negotiation.cc:407] Connection header received
0929 02:21:28.453191 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.453196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.453275 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.453361 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.454752 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.455443 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.456880 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.457313 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.457426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.460742 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.460746 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.460750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.460782 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.463218 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.463919 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.463925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.463927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.464046 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.464415 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.464420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.464423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.464757 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.464901 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.465627 (+ 726us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.465863 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:21:28.467139 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.450486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:45633 (local address 127.0.0.1:38408)
0929 02:21:28.451244 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.451287 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.452907 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.453589 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.453597 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.454061 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.454592 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.454609 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.455585 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.455590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.456745 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.456756 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.457528 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.457535 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.457731 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.458344 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.458366 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.460533 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.463352 (+ 2819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.463361 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.463377 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.463817 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.464144 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.464149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.464153 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.464316 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.464877 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.464885 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.465235 (+ 350us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.465537 (+ 302us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.466066 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":323,"threads_started":1}
I20260929 02:21:28.468689 21426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:21:28.482383 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.469393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55344)
0929 02:21:28.469571 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.469590 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.469692 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.470457 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.470462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.470880 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.471043 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.471053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.472311 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.472315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.473430 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.473441 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.474484 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.474492 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.474653 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.475209 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.475233 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.477092 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.479905 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.479910 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.479914 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.480246 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.480588 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.480593 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.480596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.480730 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.481380 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.481386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.481542 (+ 156us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.481696 (+ 154us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.481951 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:21:28.482383 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.469525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55344 (local address 127.7.170.61:36953)
0929 02:21:28.470071 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.470077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.470102 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:21:28.470143 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.470148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.470312 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.470418 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.471189 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.472177 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.473553 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.474341 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.474437 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.477275 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.477285 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.477297 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.477329 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.479789 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.480359 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.480365 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.480367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.480485 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.480827 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.480832 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.480835 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.481235 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.481430 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.481877 (+ 447us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.482068 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":266,"threads_started":1}
I20260929 02:21:28.484797 21426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:21:28.485841 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.485434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43894)
0929 02:21:28.485594 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.485706 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260929 02:21:28.486054 21378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.487483 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.487122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43910)
0929 02:21:28.487260 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.487351 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 02:21:28.487673 21426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
I20260929 02:21:28.491879 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:39361
--webserver_interface=127.7.170.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:28.536999 21426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 1
I20260929 02:21:28.538079 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.537633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43918)
0929 02:21:28.537826 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.537930 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 02:21:28.539115 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.538791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43922)
0929 02:21:28.538935 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.539007 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260929 02:21:28.539316 21426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.607533 21426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 2
I20260929 02:21:28.608687 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.608232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43934)
0929 02:21:28.608441 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.608534 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 02:21:28.609660 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.609363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43940)
0929 02:21:28.609480 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.609548 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:21:28.609860 21426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.687402 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } attempt: 5
I20260929 02:21:28.701056 21426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 3
I20260929 02:21:28.702057 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.701612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43944)
0929 02:21:28.701813 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.701901 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:21:28.703090 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.702646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43948)
0929 02:21:28.702852 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.702945 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
W20260929 02:21:28.703655 21426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.708881 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.688184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55348)
0929 02:21:28.688405 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.688427 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.688569 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.688909 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.688914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.689349 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.689545 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.689555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.690558 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.690562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.691424 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.691436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.692115 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.692120 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.692304 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.701543 (+ 9239us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.701572 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.704054 (+ 2482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.706786 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.706792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.706796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.707134 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.707452 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.707465 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.707469 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.707591 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.708069 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.708073 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.708222 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.708405 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.708619 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260929 02:21:28.709263 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.688244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55348 (local address 127.7.170.61:36953)
0929 02:21:28.688430 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.688436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.688492 (+ 56us) server_negotiation.cc:407] Connection header received
0929 02:21:28.688695 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.688699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.688776 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.688861 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.689702 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.690420 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.691562 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.691980 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.692086 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.704234 (+ 12148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.704238 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.704242 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.704274 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.706655 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.707239 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.707244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.707246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.707354 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.707683 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.707688 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.707691 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.707954 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.708087 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.708543 (+ 456us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.708760 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:21:28.710126 21352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:21:28.711026 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.710700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43964)
0929 02:21:28.710833 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.710903 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:21:28.711800 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.711514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43972)
0929 02:21:28.711655 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.711717 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:21:28.711966 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.763180 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 1
I20260929 02:21:28.764240 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.763782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43984)
0929 02:21:28.763980 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.764080 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 02:21:28.765151 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.764816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43986)
0929 02:21:28.764973 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.765042 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:21:28.765334 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
W20260929 02:21:28.804847 21433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:28.805166 21433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:28.805294 21433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:28.812346 21433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:28.812451 21433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:28.812530 21433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:28.812605 21433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:28.812664 21433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:28.822275 21433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:45633,127.7.170.61:36953,127.7.170.60:39361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:39361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.170.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:28.823539 21433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:28.824766 21433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:21:28.833549 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 2
I20260929 02:21:28.834760 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.834172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:43990)
0929 02:21:28.834402 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.834506 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260929 02:21:28.835805 21438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:28.835906 21439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:28.836134 21441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:28.836185 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.835657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44006)
0929 02:21:28.835970 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.836052 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
W20260929 02:21:28.836447 21352 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.836915 21433 server_base.cc:1034] running on GCE node
I20260929 02:21:28.837699 21433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:28.839030 21433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:28.840206 21433 hybrid_clock.cc:648] HybridClock initialized: now 1790648488840148 us; error 47 us; skew 500 ppm
I20260929 02:21:28.840560 21433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:28.843500 21433 init.cc:377] Logged in from keytab as kudu/127.7.170.60@KRBTEST.COM (short username kudu)
I20260929 02:21:28.844763 21433 webserver.cc:460] Webserver started at http://127.7.170.60:40577/ using document root <none> and password file <none>
I20260929 02:21:28.845427 21433 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:28.845523 21433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:28.845757 21433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:28.846871 21426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 4
I20260929 02:21:28.847819 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.847423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44008)
0929 02:21:28.847602 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.847679 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 02:21:28.848642 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.848372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44016)
0929 02:21:28.848482 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.848543 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 02:21:28.848817 21426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:39361: Network error: Client connection negotiation failed: client connection to 127.7.170.60:39361: connect: Connection refused (error 111)
I20260929 02:21:28.849108 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data/instance:
uuid: "027f9222770242f78ed2535bf6e41cc2"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "6ae40beee8cdc31527ef980d29429829"
server_key_iv: "82ef21971c16d9abc5ffdf644f8265d8"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.849838 21433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal/instance:
uuid: "027f9222770242f78ed2535bf6e41cc2"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "6ae40beee8cdc31527ef980d29429829"
server_key_iv: "82ef21971c16d9abc5ffdf644f8265d8"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.854611 21433 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:21:28.857739 21448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:28.859042 21433 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:28.859197 21433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "027f9222770242f78ed2535bf6e41cc2"
format_stamp: "Formatted at 2026-09-29 02:21:28 on dist-test-slave-8cs2"
server_key: "6ae40beee8cdc31527ef980d29429829"
server_key_iv: "82ef21971c16d9abc5ffdf644f8265d8"
server_key_version: "encryptionkey@0"
I20260929 02:21:28.859373 21433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:28.895435 21433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:28.899621 21433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:28.899870 21433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:28.916025 21433 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.60:39361
I20260929 02:21:28.916036 21499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.60:39361 every 8 connection(s)
I20260929 02:21:28.917471 21433 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
I20260929 02:21:28.921147 21500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:28.921602 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21433
I20260929 02:21:28.921782 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790648296482495-7848-0/minicluster-data/master-2/wal/instance
I20260929 02:21:28.922374 7848 external_mini_cluster.cc:1428] Setting key 40ce21c4c2e7e93f0dc5b2270368b203
I20260929 02:21:28.930063 21500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:28.942652 21352 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 3
I20260929 02:21:28.944783 21503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.924268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.7.170.60:39361)
0929 02:21:28.925091 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.925103 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.925145 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:21:28.927137 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.927160 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.927552 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.927878 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.929517 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.930835 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.932204 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.933278 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.933451 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.937604 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.937614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.937627 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.937671 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.940541 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.941296 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.941304 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.941313 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.941478 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.941932 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.941938 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.941942 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.942414 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.942681 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.943113 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.943500 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":377,"threads_started":1}
I20260929 02:21:28.947206 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.932408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:45633 (local address 127.0.0.1:38422)
0929 02:21:28.933114 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.933158 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.933311 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.933761 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.933768 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.934479 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.934802 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.934814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.935806 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.935811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.936975 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.936998 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.937891 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.937895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.938031 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.938663 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.938681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.940985 (+ 2304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.944327 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.944339 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.944348 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.944923 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.945342 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.945345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.945348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.945472 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.946112 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.946119 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.946316 (+ 197us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.946515 (+ 199us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.946793 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":340,"threads_started":1}
I20260929 02:21:28.947212 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.932344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.7.170.62:45633)
0929 02:21:28.932489 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.932494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.933258 (+ 764us) server_negotiation.cc:407] Connection header received
0929 02:21:28.933489 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.933494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.933580 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.933676 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.934961 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.935661 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.937153 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.937721 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.937849 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.941144 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.941149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.941152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.941184 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.944172 (+ 2988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.945095 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.945099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.945102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.945234 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.945591 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.945595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.945598 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.945901 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.946044 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.946634 (+ 590us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.946843 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":47}
I20260929 02:21:28.948750 21500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:21:28.958447 21354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.943169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44028)
0929 02:21:28.943524 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.943543 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.943636 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.945172 (+ 1536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.945177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.945638 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.945836 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.945847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.947433 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.947438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.948783 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.948794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.949855 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.949861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.950031 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.950651 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.950673 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.953035 (+ 2362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.956240 (+ 3205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.956245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.956249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.956587 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.956938 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.956943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.956946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.957073 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.957544 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.957549 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.957697 (+ 148us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.957835 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.958087 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260929 02:21:28.958767 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.943936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.7.170.60:39361)
0929 02:21:28.944610 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.944615 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.944635 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:21:28.944682 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.944686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.944752 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.944861 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.946034 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.947271 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.948964 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.949688 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.949874 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.953220 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.953226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.953231 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.953264 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.956113 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.956693 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.956702 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.956705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.956826 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.957165 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.957170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.957174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.957441 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.957580 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.957985 (+ 405us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.958184 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":335,"threads_started":1}
I20260929 02:21:28.963651 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.949380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36953 (local address 127.0.0.1:55360)
0929 02:21:28.950514 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.950531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.950626 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.950898 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.950901 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.951223 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.951363 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.951371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.952667 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.952672 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.953728 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.953742 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.954779 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.954785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.955047 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.955561 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.955583 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.958351 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.961376 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.961382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.961386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.961722 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.962127 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.962132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.962136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.962262 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.962742 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.962747 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.962887 (+ 140us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.963054 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.963285 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010}
I20260929 02:21:28.963677 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.949547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55360 (local address 127.7.170.61:36953)
0929 02:21:28.949721 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.949727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.950596 (+ 869us) server_negotiation.cc:407] Connection header received
0929 02:21:28.950720 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.950726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.950791 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.950870 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.951510 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.952512 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.953893 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.954449 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.954543 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.958565 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.958571 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.958575 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.958612 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.961266 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.961872 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.961877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.961881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.962030 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.962357 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.962361 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.962364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.962623 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.962751 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.963147 (+ 396us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.963341 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:21:28.964726 21500 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:21:28.965024 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.948147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38430 (local address 127.7.170.62:45633)
0929 02:21:28.948631 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.948636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.948656 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:21:28.948701 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.948709 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.948771 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.948863 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.950345 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.951421 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.953066 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.953656 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.953779 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.958747 (+ 4968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.958753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.958757 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.958788 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.962034 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.962795 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.962800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.962803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.962948 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.963352 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.963357 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.963360 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.963707 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.963949 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.964464 (+ 515us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.964674 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373}
I20260929 02:21:28.966249 21352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: Bootstrap starting.
I20260929 02:21:28.969992 21352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:28.971277 21352 log.cc:824] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: Log is configured to *not* fsync() on all Append() calls
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
I20260929 02:21:28.974655 21352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: No bootstrap required, opened a new log
Sep 29 02:21:28 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:21:28.977870 21504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.965357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44042)
0929 02:21:28.965488 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:28.965505 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:28.965594 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:28.965864 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:28.965868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:28.966264 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:28.966428 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.966439 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.967570 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.967575 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.968386 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:28.968397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.969288 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.969295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:28.969470 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.969997 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:28.970019 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:28.972391 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:28.975669 (+ 3278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.975674 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.975677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.976143 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.976454 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:28.976459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:28.976462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:28.976573 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:28.977010 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:28.977014 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:28.977136 (+ 122us) client_negotiation.cc:769] Sending connection context
0929 02:21:28.977243 (+ 107us) client_negotiation.cc:240] Negotiation successful
0929 02:21:28.977503 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260929 02:21:28.978142 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.965515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44042 (local address 127.7.170.60:39361)
0929 02:21:28.965633 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.965639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.965652 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:21:28.965689 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.965693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.965751 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.965835 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.966584 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.967460 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.968511 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.969129 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.969243 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.972562 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.972566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.972569 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.972594 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.975533 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.976238 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.976243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.976247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.976361 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.976664 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.976669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.976672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.976897 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.977040 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.977335 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.977652 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":42}
I20260929 02:21:28.981819 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:28.966595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55376 (local address 127.7.170.61:36953)
0929 02:21:28.966742 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:28.966747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:28.966867 (+ 120us) server_negotiation.cc:407] Connection header received
0929 02:21:28.967004 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:28.967009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:28.967072 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:28.967160 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:28.968578 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.969510 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.970826 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:28.971223 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:28.971332 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:28.975556 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:28.975561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:28.975564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:28.975591 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:28.978822 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.979594 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.979599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.979603 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.979751 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:28.980161 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:28.980166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:28.980169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:28.980494 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:28.980722 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:28.981283 (+ 561us) server_negotiation.cc:299] Negotiation successful
0929 02:21:28.981507 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:21:28.981992 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: Bootstrap starting.
I20260929 02:21:28.985038 21352 raft_consensus.cc:348] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:28.985450 21352 raft_consensus.cc:374] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:28.985569 21352 raft_consensus.cc:729] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e178e35dd7a949199cedf4b0e849d91a, State: Initialized, Role: FOLLOWER
I20260929 02:21:28.986250 21352 consensus_queue.cc:260] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:28.987156 21500 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:28.987262 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:21:28.988441 21500 log.cc:824] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:28.993180 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:28.993610 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:28.995366 21352 sys_catalog.cc:564] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:28.999848 21500 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: No bootstrap required, opened a new log
I20260929 02:21:29.011113 21500 raft_consensus.cc:348] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.011610 21500 raft_consensus.cc:374] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:29.011729 21500 raft_consensus.cc:729] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 027f9222770242f78ed2535bf6e41cc2, State: Initialized, Role: FOLLOWER
I20260929 02:21:29.012550 21500 consensus_queue.cc:260] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.019507 21521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.019898 21521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.021049 21500 sys_catalog.cc:564] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:29.039143 21355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:29.007457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.7.170.62:45633)
0929 02:21:29.007604 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:29.007610 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:29.018402 (+ 10792us) server_negotiation.cc:407] Connection header received
0929 02:21:29.019766 (+ 1364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:29.019773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:29.019862 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:29.019937 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:29.024398 (+ 4461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.025499 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.027826 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.028460 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.028554 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:29.032214 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:29.032221 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:29.032226 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:29.032284 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:29.035825 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.036697 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.036703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.036707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.036899 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.037358 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.037364 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.037367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.037704 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:29.037924 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:29.038513 (+ 589us) server_negotiation.cc:299] Negotiation successful
0929 02:21:29.038726 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:21:29.043994 21430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:29.007409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55388 (local address 127.7.170.61:36953)
0929 02:21:29.007605 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:29.007610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:29.020184 (+ 12574us) server_negotiation.cc:407] Connection header received
0929 02:21:29.020618 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:29.020624 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:29.020703 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:29.020773 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:29.022754 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.023861 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.028054 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.028664 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.028749 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:29.037074 (+ 8325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:29.037080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:29.037085 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:29.037126 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:29.040384 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.041611 (+ 1227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.041616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.041620 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.041774 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.042309 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.042314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.042317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.042701 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:29.042913 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:29.043432 (+ 519us) server_negotiation.cc:299] Negotiation successful
0929 02:21:29.043666 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260929 02:21:29.049362 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:29.018995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44050 (local address 127.7.170.60:39361)
0929 02:21:29.019150 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:29.019156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:29.024060 (+ 4904us) server_negotiation.cc:407] Connection header received
0929 02:21:29.024227 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:29.024232 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:29.024306 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:29.024384 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:29.029571 (+ 5187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.030689 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.034629 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.035301 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.035400 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:29.043114 (+ 7714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:29.043120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:29.043125 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:29.043164 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:29.046623 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.047262 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.047269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.047272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.047446 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.047921 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.047928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.047931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.048287 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:29.048599 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:29.048814 (+ 215us) server_negotiation.cc:299] Negotiation successful
0929 02:21:29.049039 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:21:29.067155 21507 raft_consensus.cc:482] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:21:29.067446 21507 raft_consensus.cc:504] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.057585 21285 master_runner.cc:186] Error getting master registration for 127.7.170.61:36953: OK, instance_id { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" instance_seqno: 1790648488432071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:21:29.068349 21285 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:21:29.069067 21507 leader_election.cc:288] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6966aa130db646118ccf5a14a8b2d46c (127.7.170.61:36953), 027f9222770242f78ed2535bf6e41cc2 (127.7.170.60:39361)
I20260929 02:21:29.069850 21475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178e35dd7a949199cedf4b0e849d91a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "027f9222770242f78ed2535bf6e41cc2" is_pre_election: true
I20260929 02:21:29.070447 21475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e178e35dd7a949199cedf4b0e849d91a in term 0.
I20260929 02:21:29.071590 21304 leader_election.cc:302] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 027f9222770242f78ed2535bf6e41cc2, e178e35dd7a949199cedf4b0e849d91a; no voters:
I20260929 02:21:29.071969 21507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:21:29.072115 21507 raft_consensus.cc:482] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:21:29.072224 21507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:21:29.072230 21532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:21:29.072365 21532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:21:29.075348 21507 raft_consensus.cc:504] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.075980 21507 leader_election.cc:288] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [CANDIDATE]: Term 1 election: Requested vote from peers 6966aa130db646118ccf5a14a8b2d46c (127.7.170.61:36953), 027f9222770242f78ed2535bf6e41cc2 (127.7.170.60:39361)
W20260929 02:21:29.079124 21539 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:21:29.079242 21539 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:21:29.081151 21475 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178e35dd7a949199cedf4b0e849d91a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "027f9222770242f78ed2535bf6e41cc2"
I20260929 02:21:29.081364 21475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:29.084753 21475 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e178e35dd7a949199cedf4b0e849d91a in term 1.
I20260929 02:21:29.086478 21304 leader_election.cc:302] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 027f9222770242f78ed2535bf6e41cc2, e178e35dd7a949199cedf4b0e849d91a; no voters:
I20260929 02:21:29.076157 21433 master_runner.cc:186] Error getting master registration for 127.7.170.61:36953: OK, instance_id { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" instance_seqno: 1790648488432071 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:21:29.086784 21507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:29.087397 21433 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:21:29.087528 21507 raft_consensus.cc:686] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [term 1 LEADER]: Becoming Leader. State: Replica: e178e35dd7a949199cedf4b0e849d91a, State: Running, Role: LEADER
I20260929 02:21:29.088169 21507 consensus_queue.cc:237] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.106950 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178e35dd7a949199cedf4b0e849d91a. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.107575 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:29.114518 21543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:29.118637 21543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:29.128331 21475 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [term 1 FOLLOWER]: Refusing update from remote peer e178e35dd7a949199cedf4b0e849d91a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:29.130120 21507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [LEADER]: Connected to new peer: Peer: permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:29.147989 21426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } attempt: 5
I20260929 02:21:29.149248 21521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178e35dd7a949199cedf4b0e849d91a. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.149462 21521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.151369 21544 mvcc.cc:205] Tried to move back new op lower bound from 7334496211457748992 to 7334496211310075904. Current Snapshot: MvccSnapshot[applied={T|T < 7334496211457748992}]
W20260929 02:21:29.151979 21454 tablet.cc:2369] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:21:29.157883 21541 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.158151 21541 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: This master's current role is: LEADER
Sep 29 02:21:29 dist-test-slave-8cs2 krb5kdc[21272](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
I20260929 02:21:29.161664 21545 mvcc.cc:205] Tried to move back new op lower bound from 7334496211457748992 to 7334496211310075904. Current Snapshot: MvccSnapshot[applied={T|T < 7334496211457748992}]
I20260929 02:21:29.163163 21543 catalog_manager.cc:1347] Generated new cluster ID: 692aef4d9ae14f29b35cea67ebbad579
I20260929 02:21:29.163265 21543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:29.163738 21521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.163972 21521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.168357 21428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:29.148538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:39361 (local address 127.0.0.1:44056)
0929 02:21:29.148726 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:29.148747 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:29.148856 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:29.150431 (+ 1575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:29.150436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:29.150935 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:29.151140 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:29.151153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.153785 (+ 2632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.153792 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:29.155270 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:29.155287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.156631 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.156636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:29.156859 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:29.157629 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:29.157651 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:29.160190 (+ 2539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:29.164549 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:29.164554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:29.164557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:29.165076 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:29.165875 (+ 799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:29.165880 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:29.165883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:29.166050 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:29.167387 (+ 1337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:29.167392 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:29.167557 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 02:21:29.167700 (+ 143us) client_negotiation.cc:240] Negotiation successful
0929 02:21:29.167996 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 02:21:29.172748 21505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:29.149749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44056 (local address 127.7.170.60:39361)
0929 02:21:29.149974 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:29.149980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:29.149997 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:21:29.150045 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:29.150049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:29.150119 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:29.150206 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:29.151810 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.153102 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.155509 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:29.156251 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:29.156364 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:29.160387 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:29.160394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:29.160399 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:29.160435 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:29.164442 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.165280 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.165287 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.165290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.165644 (+ 354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:29.166262 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:29.166267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:29.166307 (+ 40us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:29.167152 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:29.171158 (+ 4006us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:29.171434 (+ 276us) server_negotiation.cc:299] Negotiation successful
0929 02:21:29.171923 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260929 02:21:29.178072 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: Bootstrap starting.
I20260929 02:21:29.181602 21426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:29.182713 21426 log.cc:824] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:29.194250 21543 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:21:29.195847 21426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: No bootstrap required, opened a new log
I20260929 02:21:29.196005 21543 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:29.210366 21543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a: Generated new TSK 0
I20260929 02:21:29.211580 21543 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:21:29.216457 21426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.216895 21426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:29.217047 21426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6966aa130db646118ccf5a14a8b2d46c, State: Initialized, Role: FOLLOWER
I20260929 02:21:29.217732 21426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } }
I20260929 02:21:29.218706 21550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.219135 21550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.220000 21426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:29.221418 21401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178e35dd7a949199cedf4b0e849d91a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6966aa130db646118ccf5a14a8b2d46c" is_pre_election: true
I20260929 02:21:29.222818 21400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e178e35dd7a949199cedf4b0e849d91a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6966aa130db646118ccf5a14a8b2d46c"
I20260929 02:21:29.223448 21399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:29.227087 21399 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [term 1 FOLLOWER]: Refusing update from remote peer e178e35dd7a949199cedf4b0e849d91a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:29.228366 21507 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:29.262430 21550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e178e35dd7a949199cedf4b0e849d91a. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.262779 21550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.263633 21555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.269466 21555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:21:29.327389 21571 catalog_manager.cc:1259] Loaded cluster ID: 692aef4d9ae14f29b35cea67ebbad579
I20260929 02:21:29.327481 21571 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: loading cluster ID for follower catalog manager: success
I20260929 02:21:29.334573 21571 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: acquiring CA information for follower catalog manager: success
I20260929 02:21:29.337033 21571 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6966aa130db646118ccf5a14a8b2d46c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:21:29.664139 21507 sys_catalog.cc:455] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e178e35dd7a949199cedf4b0e849d91a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e178e35dd7a949199cedf4b0e849d91a" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45633 } } peers { permanent_uuid: "6966aa130db646118ccf5a14a8b2d46c" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36953 } } peers { permanent_uuid: "027f9222770242f78ed2535bf6e41cc2" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 39361 } } }
I20260929 02:21:29.664517 21507 sys_catalog.cc:458] T 00000000000000000000000000000000 P e178e35dd7a949199cedf4b0e849d91a [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:30.081282 21539 catalog_manager.cc:1259] Loaded cluster ID: 692aef4d9ae14f29b35cea67ebbad579
I20260929 02:21:30.081383 21539 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: loading cluster ID for follower catalog manager: success
I20260929 02:21:30.083863 21539 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: acquiring CA information for follower catalog manager: success
I20260929 02:21:30.085942 21539 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 027f9222770242f78ed2535bf6e41cc2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:21:30.365422 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21285
I20260929 02:21:30.366567 21347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:30.476981 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21285
I20260929 02:21:30.494724 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21359
I20260929 02:21:30.495805 21421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:30.599423 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21359
I20260929 02:21:30.615952 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21433
I20260929 02:21:30.617106 21495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:30.724377 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21433
2026-09-29T02:21:30Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4382 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:21:30 dist-test-slave-8cs2 krb5kdc[21579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:21:30 dist-test-slave-8cs2 krb5kdc[21579](info): set up 2 sockets
Sep 29 02:21:30 dist-test-slave-8cs2 krb5kdc[21579](info): commencing operation
krb5kdc: starting...
W20260929 02:21:31.737458 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.974s user 0.003s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:21:31 dist-test-slave-8cs2 krb5kdc[21579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:21:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:21:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:21:31.972532 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44657
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:34923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:44657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:32.289234 21592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:32.289525 21592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:32.289620 21592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:32.296626 21592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:32.296725 21592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:32.296779 21592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:32.296834 21592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:32.296876 21592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:32.306578 21592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:34923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:44657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:32.307835 21592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:32.309096 21592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:32.320176 21600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:32.320211 21598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:32.320245 21597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:32.321017 21592 server_base.cc:1034] running on GCE node
I20260929 02:21:32.321762 21592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:32.323148 21592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:32.324338 21592 hybrid_clock.cc:648] HybridClock initialized: now 1790648492324285 us; error 40 us; skew 500 ppm
I20260929 02:21:32.324661 21592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:32 dist-test-slave-8cs2 krb5kdc[21579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:32.327525 21592 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:32.328998 21592 webserver.cc:460] Webserver started at http://127.7.170.62:33235/ using document root <none> and password file <none>
I20260929 02:21:32.329654 21592 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:32.329764 21592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:32.330054 21592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:32.332962 21592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094"
format_stamp: "Formatted at 2026-09-29 02:21:32 on dist-test-slave-8cs2"
server_key: "db225bec9ea3fa47e3051ed0b938f85d"
server_key_iv: "d73ad9f63d414217d0b9b4305a88b133"
server_key_version: "encryptionkey@0"
I20260929 02:21:32.333689 21592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094"
format_stamp: "Formatted at 2026-09-29 02:21:32 on dist-test-slave-8cs2"
server_key: "db225bec9ea3fa47e3051ed0b938f85d"
server_key_iv: "d73ad9f63d414217d0b9b4305a88b133"
server_key_version: "encryptionkey@0"
I20260929 02:21:32.338593 21592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:21:32.341879 21607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:32.343147 21592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:21:32.343297 21592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094"
format_stamp: "Formatted at 2026-09-29 02:21:32 on dist-test-slave-8cs2"
server_key: "db225bec9ea3fa47e3051ed0b938f85d"
server_key_iv: "d73ad9f63d414217d0b9b4305a88b133"
server_key_version: "encryptionkey@0"
I20260929 02:21:32.343464 21592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:32.368992 21592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:32.372648 21592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:32.372893 21592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:32.389710 21592 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:44657
I20260929 02:21:32.389740 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:44657 every 8 connection(s)
I20260929 02:21:32.391039 21592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:32.393913 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21592
I20260929 02:21:32.394124 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:21:32.394721 7848 external_mini_cluster.cc:1428] Setting key f10871c6b489d06dc92f34fa9312d277
I20260929 02:21:32.394801 21659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:32.408907 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094: Bootstrap starting.
Sep 29 02:21:32 dist-test-slave-8cs2 krb5kdc[21579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:32.412573 21659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:32.413890 21659 log.cc:824] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:32.417209 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094: No bootstrap required, opened a new log
I20260929 02:21:32.418918 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:32.396465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35728 (local address 127.7.170.62:44657)
0929 02:21:32.397242 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:32.397256 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:32.397304 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:21:32.399642 (+ 2338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:32.399674 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:32.400210 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:32.400676 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:32.401956 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:32.403907 (+ 1951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:32.405224 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:32.406319 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:32.406509 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:32.410530 (+ 4021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:32.410541 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:32.410560 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:32.410608 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:32.414309 (+ 3701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:32.415207 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:32.415215 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:32.415223 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:32.415424 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:32.415781 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:32.415786 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:32.415789 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:32.416451 (+ 662us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:32.416741 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:32.417198 (+ 457us) server_negotiation.cc:299] Negotiation successful
0929 02:21:32.417628 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":296,"threads_started":1}
I20260929 02:21:32.422748 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:21:32.425624 21659 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } }
I20260929 02:21:32.425994 21659 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:32.426085 21659 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2594f3ecb6c494f8f5aa1aaf0e33094, State: Initialized, Role: FOLLOWER
I20260929 02:21:32.426622 21659 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } }
I20260929 02:21:32.426836 21659 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:32.426954 21659 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:32.427103 21659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:32.429255 21659 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } }
I20260929 02:21:32.429698 21659 leader_election.cc:302] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2594f3ecb6c494f8f5aa1aaf0e33094; no voters:
I20260929 02:21:32.430217 21659 leader_election.cc:288] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:21:32.430359 21664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:32.430961 21664 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [term 1 LEADER]: Becoming Leader. State: Replica: a2594f3ecb6c494f8f5aa1aaf0e33094, State: Running, Role: LEADER
I20260929 02:21:32.431459 21664 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } }
I20260929 02:21:32.432062 21659 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:32.434202 21665 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } } }
I20260929 02:21:32.434696 21665 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:32.434229 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2594f3ecb6c494f8f5aa1aaf0e33094. Latest consensus state: current_term: 1 leader_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2594f3ecb6c494f8f5aa1aaf0e33094" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44657 } } }
I20260929 02:21:32.435117 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:32.436168 21671 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:32.443655 21671 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:32.451620 21671 catalog_manager.cc:1347] Generated new cluster ID: cb07332266a34100a94a024634b86279
I20260929 02:21:32.451735 21671 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:32.472345 21671 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:21:32.473912 21671 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:32.483529 21671 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2594f3ecb6c494f8f5aa1aaf0e33094: Generated new TSK 0
I20260929 02:21:32.484334 21671 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:21:32.534252 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21592
I20260929 02:21:32.535405 21654 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:32.631408 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21592
2026-09-29T02:21:32Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1902 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:21:32 dist-test-slave-8cs2 krb5kdc[21687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:21:32 dist-test-slave-8cs2 krb5kdc[21687](info): set up 2 sockets
Sep 29 02:21:32 dist-test-slave-8cs2 krb5kdc[21687](info): commencing operation
krb5kdc: starting...
W20260929 02:21:33.653622 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.987s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:21:33 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:21:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:21:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:21:33.885838 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:34.207828 21700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:34.208137 21700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:34.208266 21700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:34.215371 21700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:34.215488 21700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:34.215564 21700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:34.215636 21700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:34.215698 21700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:34.225502 21700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:34.226810 21700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:34.228089 21700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:34.239064 21708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:34.239171 21706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:34.239195 21705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:34.245360 21700 server_base.cc:1034] running on GCE node
I20260929 02:21:34.246130 21700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:34.247483 21700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:34.248711 21700 hybrid_clock.cc:648] HybridClock initialized: now 1790648494248660 us; error 35 us; skew 500 ppm
I20260929 02:21:34.249091 21700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:34 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648494, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:34.262856 21700 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:34.264428 21700 webserver.cc:460] Webserver started at http://127.7.170.62:36381/ using document root <none> and password file <none>
I20260929 02:21:34.265202 21700 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:34.265328 21700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:34.265625 21700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:34.268580 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e742251329a84c839c44af920ccf4baf"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "d2c1ead3bc5a37eede10aa11bdede8cb"
server_key_iv: "0dbc23d4159f394968f6f260ecc744fc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.269320 21700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e742251329a84c839c44af920ccf4baf"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "d2c1ead3bc5a37eede10aa11bdede8cb"
server_key_iv: "0dbc23d4159f394968f6f260ecc744fc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.274243 21700 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:21:34.277480 21715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:34.278738 21700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:21:34.278898 21700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e742251329a84c839c44af920ccf4baf"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "d2c1ead3bc5a37eede10aa11bdede8cb"
server_key_iv: "0dbc23d4159f394968f6f260ecc744fc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.279109 21700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:34.293684 21700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:34.297505 21700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:34.297744 21700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:34.314316 21700 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38141
I20260929 02:21:34.314337 21766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38141 every 8 connection(s)
I20260929 02:21:34.315585 21700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:34.318910 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21700
I20260929 02:21:34.319125 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:21:34.319317 21767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:34.319947 7848 external_mini_cluster.cc:1428] Setting key f8ebc0f996701dc4f43a803b97c7c2e1
I20260929 02:21:34.333014 21767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap starting.
Sep 29 02:21:34 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:34.336325 21767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:34.337404 21767 log.cc:824] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:34.340447 21767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: No bootstrap required, opened a new log
I20260929 02:21:34.342253 21770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:34.321908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54646 (local address 127.7.170.62:38141)
0929 02:21:34.322669 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:34.322680 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:34.322719 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:21:34.324169 (+ 1450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:34.324188 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:34.324546 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:34.324859 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:34.326550 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.327766 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.329596 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.330283 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.330422 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:34.335001 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:34.335012 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:34.335025 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:34.335059 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:34.338370 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:34.339139 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:34.339145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:34.339151 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:34.339309 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:34.339708 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:34.339713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:34.339716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:34.340232 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:34.340457 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:34.340870 (+ 413us) server_negotiation.cc:299] Negotiation successful
0929 02:21:34.341297 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":375,"threads_started":1}
I20260929 02:21:34.347921 21767 raft_consensus.cc:348] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:34.348243 21767 raft_consensus.cc:374] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:34.348340 21767 raft_consensus.cc:729] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e742251329a84c839c44af920ccf4baf, State: Initialized, Role: FOLLOWER
I20260929 02:21:34.348824 21767 consensus_queue.cc:260] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:34.349073 21767 raft_consensus.cc:388] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:34.349190 21767 raft_consensus.cc:482] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:34.349331 21767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:34.351377 21767 raft_consensus.cc:504] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:34.351826 21767 leader_election.cc:302] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e742251329a84c839c44af920ccf4baf; no voters:
I20260929 02:21:34.352385 21767 leader_election.cc:288] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:21:34.352506 21772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:34.353113 21772 raft_consensus.cc:686] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 LEADER]: Becoming Leader. State: Replica: e742251329a84c839c44af920ccf4baf, State: Running, Role: LEADER
I20260929 02:21:34.353621 21772 consensus_queue.cc:237] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:34.354205 21767 sys_catalog.cc:564] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:34.356150 21774 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e742251329a84c839c44af920ccf4baf. Latest consensus state: current_term: 1 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:34.356526 21774 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:34.357632 21773 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:34.357874 21773 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:34.362136 21781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:34.366999 21781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:34.375404 21781 catalog_manager.cc:1347] Generated new cluster ID: 9695d43113694c0897d04e66d5ca8b25
I20260929 02:21:34.375515 21781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:34.399881 21781 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:21:34.401330 21781 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:34.413000 21781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Generated new TSK 0
I20260929 02:21:34.413802 21781 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:21:34.481065 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:34.823477 21793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:34.823770 21793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:34.823896 21793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:34.831528 21793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:34.831653 21793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:34.831812 21793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:34.842161 21793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:34.843506 21793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:34.844810 21793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:34.857300 21798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:34.857324 21799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:34.857534 21801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:34.858806 21793 server_base.cc:1034] running on GCE node
I20260929 02:21:34.859398 21793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:34.860323 21793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:34.861521 21793 hybrid_clock.cc:648] HybridClock initialized: now 1790648494861470 us; error 38 us; skew 500 ppm
I20260929 02:21:34.861897 21793 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:34 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648494, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:34.865116 21793 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:34.866701 21793 webserver.cc:460] Webserver started at http://127.7.170.1:37067/ using document root <none> and password file <none>
I20260929 02:21:34.867394 21793 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:34.867507 21793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:34.867782 21793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:34.870790 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "37d1d4296f284fe6b440fbe3062779bc"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "a7ddb116109ce89e1652964e10c6ac56"
server_key_iv: "60686093683b0e84b43f8e54180732cc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.871536 21793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "37d1d4296f284fe6b440fbe3062779bc"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "a7ddb116109ce89e1652964e10c6ac56"
server_key_iv: "60686093683b0e84b43f8e54180732cc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.876552 21793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:21:34.879928 21808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:34.881312 21793 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260929 02:21:34.881472 21793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "37d1d4296f284fe6b440fbe3062779bc"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "a7ddb116109ce89e1652964e10c6ac56"
server_key_iv: "60686093683b0e84b43f8e54180732cc"
server_key_version: "encryptionkey@0"
I20260929 02:21:34.881662 21793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:34.911242 21793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:34.915318 21793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:34.915596 21793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:34.916522 21793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:34.917878 21793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:34.917958 21793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:34.918077 21793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:34.918136 21793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:34.948004 21793 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:45531
I20260929 02:21:34.948045 21920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:45531 every 8 connection(s)
I20260929 02:21:34.949314 21793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:34.954634 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21793
I20260929 02:21:34.954823 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:21:34.955580 7848 external_mini_cluster.cc:1428] Setting key 8df79b3c3ab6c2b43c78bc643aec867c
Sep 29 02:21:34 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648494, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:34.970245 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:34.951840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:46201 (local address 127.7.170.62:38141)
0929 02:21:34.952276 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:34.952280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:34.953295 (+ 1015us) server_negotiation.cc:407] Connection header received
0929 02:21:34.955186 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:34.955193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:34.955280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:34.955426 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:34.957001 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.957846 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.959981 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.960368 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.960463 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:34.964350 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:34.964355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:34.964358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:34.964388 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:34.966974 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:34.967700 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:34.967706 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:34.967709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:34.967845 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:34.968312 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:34.968317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:34.968320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:34.968653 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:34.968791 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:34.969658 (+ 867us) server_negotiation.cc:299] Negotiation successful
0929 02:21:34.969881 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260929 02:21:34.971009 21924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:34.952410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:46201)
0929 02:21:34.953140 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:34.953181 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:34.954920 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:34.955605 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:34.955613 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:34.956217 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:34.956837 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:34.956860 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.958023 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.958029 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:34.959845 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:34.959859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:34.960575 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:34.960581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:34.960851 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:34.961546 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:34.961572 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:34.964138 (+ 2566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:34.967133 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:34.967140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:34.967154 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:34.967573 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:34.967993 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:34.967997 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:34.967999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:34.968194 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:34.968791 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:34.968797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:34.969255 (+ 458us) client_negotiation.cc:769] Sending connection context
0929 02:21:34.969498 (+ 243us) client_negotiation.cc:240] Negotiation successful
0929 02:21:34.969904 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":257,"threads_started":1}
I20260929 02:21:34.972694 21921 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:34.973065 21921 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:34.973788 21921 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:34.976101 21732 ts_manager.cc:194] Registered new tserver with Master: 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1:45531)
I20260929 02:21:34.978164 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:46201
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:21:35.010506 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:35.342465 21927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:35.342741 21927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:35.342855 21927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:35.350314 21927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:35.350440 21927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:35.350592 21927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:35.360662 21927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:35.362003 21927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:35.363291 21927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:35.376201 21935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:35.376276 21932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:35.376314 21933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:35.376945 21927 server_base.cc:1034] running on GCE node
I20260929 02:21:35.377581 21927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:35.378588 21927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:35.379815 21927 hybrid_clock.cc:648] HybridClock initialized: now 1790648495379761 us; error 43 us; skew 500 ppm
I20260929 02:21:35.380193 21927 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:35 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648495, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:35.383452 21927 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:35.384866 21927 webserver.cc:460] Webserver started at http://127.7.170.2:38215/ using document root <none> and password file <none>
I20260929 02:21:35.385603 21927 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:35.385701 21927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:35.385978 21927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:35.389142 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "74306fd30fa03ca3871eec5219c4ae10"
server_key_iv: "d12d002715f5d70e8ec38cc3f5307629"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.389907 21927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "74306fd30fa03ca3871eec5219c4ae10"
server_key_iv: "d12d002715f5d70e8ec38cc3f5307629"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.395089 21927 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 02:21:35.398598 21942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.400070 21927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:21:35.400224 21927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "74306fd30fa03ca3871eec5219c4ae10"
server_key_iv: "d12d002715f5d70e8ec38cc3f5307629"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.400439 21927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:35.412808 21927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:35.416922 21927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:35.417196 21927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:35.418076 21927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:35.419302 21927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:35.419371 21927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.419457 21927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:35.419498 21927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.448720 21927 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:32909
I20260929 02:21:35.448760 22054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:32909 every 8 connection(s)
I20260929 02:21:35.450034 21927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:35.453227 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 21927
I20260929 02:21:35.453385 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:21:35.453899 7848 external_mini_cluster.cc:1428] Setting key 5e1a45f9258a1689ad34c67833ee843a
Sep 29 02:21:35 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648495, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:35.471410 21923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:35.452729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:51203 (local address 127.7.170.62:38141)
0929 02:21:35.452922 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:35.452949 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:35.454406 (+ 1457us) server_negotiation.cc:407] Connection header received
0929 02:21:35.456619 (+ 2213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:35.456625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:35.456709 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:35.456805 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:35.458375 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.459347 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.460917 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.461558 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.461671 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:35.465428 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:35.465434 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:35.465437 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:35.465472 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:35.468031 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:35.469010 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:35.469014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:35.469016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:35.469152 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:35.469635 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:35.469639 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:35.469641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:35.469945 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:35.470057 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:35.470959 (+ 902us) server_negotiation.cc:299] Negotiation successful
0929 02:21:35.471154 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:21:35.472751 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:35.453324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:51203)
0929 02:21:35.454189 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:35.454240 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:35.456343 (+ 2103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:35.457042 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:35.457051 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:35.457614 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:35.458225 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:35.458244 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.459503 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.459507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:35.460773 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:35.460785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.461792 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.461799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:35.462016 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:35.462653 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:35.462677 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:35.465161 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:35.468176 (+ 3015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:35.468184 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:35.468202 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:35.468796 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:35.469260 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:35.469264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:35.469268 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:35.469518 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:35.470068 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:35.470076 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:35.470517 (+ 441us) client_negotiation.cc:769] Sending connection context
0929 02:21:35.470820 (+ 303us) client_negotiation.cc:240] Negotiation successful
0929 02:21:35.471388 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":291,"threads_started":1}
I20260929 02:21:35.474382 22055 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:35.474756 22055 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:35.475677 22055 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:35.477502 21732 ts_manager.cc:194] Registered new tserver with Master: 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909)
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:21:35.478659 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:51203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:21:35.508273 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:35.841359 22060 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:35.841660 22060 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:35.841763 22060 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:35.849023 22060 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:35.849138 22060 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:35.849254 22060 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:35.859292 22060 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:35.860610 22060 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:35.861931 22060 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:35.874650 22065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:35.874738 22066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:35.874792 22068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:35.875802 22060 server_base.cc:1034] running on GCE node
I20260929 02:21:35.876396 22060 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:35.877362 22060 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:35.878556 22060 hybrid_clock.cc:648] HybridClock initialized: now 1790648495878494 us; error 60 us; skew 500 ppm
I20260929 02:21:35.878849 22060 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:35 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648495, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:35.881809 22060 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:35.883285 22060 webserver.cc:460] Webserver started at http://127.7.170.3:46837/ using document root <none> and password file <none>
I20260929 02:21:35.883955 22060 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:35.884064 22060 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:35.884344 22060 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:35.887311 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "e9aed72b1efc455e80bc50666a159a81"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "4dff5e9a17a1c53a62bbdc66704a134c"
server_key_iv: "2edb6ffe41e873b8e3e3e405b0a744a1"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.888032 22060 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "e9aed72b1efc455e80bc50666a159a81"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "4dff5e9a17a1c53a62bbdc66704a134c"
server_key_iv: "2edb6ffe41e873b8e3e3e405b0a744a1"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.892905 22060 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:21:35.896242 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.897578 22060 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:35.897746 22060 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e9aed72b1efc455e80bc50666a159a81"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "4dff5e9a17a1c53a62bbdc66704a134c"
server_key_iv: "2edb6ffe41e873b8e3e3e405b0a744a1"
server_key_version: "encryptionkey@0"
I20260929 02:21:35.897928 22060 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:35.937973 22060 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:35.941857 22060 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:35.942124 22060 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:35.943029 22060 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:35.944303 22060 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:35.944376 22060 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.944495 22060 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:35.944545 22060 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:35.974826 22060 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33693
I20260929 02:21:35.974886 22187 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33693 every 8 connection(s)
I20260929 02:21:35.976043 22060 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:21:35.981204 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22060
I20260929 02:21:35.981206 21921 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:35.981386 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:21:35.981896 7848 external_mini_cluster.cc:1428] Setting key 67d574b03d8bef104891f64c5a603966
Sep 29 02:21:35 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648495, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:35.995409 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:35.978443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:59279 (local address 127.7.170.62:38141)
0929 02:21:35.978879 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:35.978883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:35.979841 (+ 958us) server_negotiation.cc:407] Connection header received
0929 02:21:35.981840 (+ 1999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:35.981845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:35.981928 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:35.982034 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:35.983782 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.984879 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.986465 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.986848 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.986942 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:35.990062 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:35.990067 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:35.990070 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:35.990102 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:35.992485 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:35.993204 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:35.993208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:35.993210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:35.993332 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:35.993745 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:35.993748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:35.993750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:35.993999 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:35.994139 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:35.994813 (+ 674us) server_negotiation.cc:299] Negotiation successful
0929 02:21:35.995030 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":221,"threads_started":1}
I20260929 02:21:35.996449 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:35.979014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:59279)
0929 02:21:35.979685 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:35.979721 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:35.981560 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:35.982350 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:35.982358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:35.983035 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:35.983632 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:35.983656 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.985033 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.985036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:35.986353 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:35.986364 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:35.987016 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:35.987021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:35.987205 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:35.987748 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:35.987769 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:35.989857 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:35.992617 (+ 2760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:35.992626 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:35.992641 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:35.993092 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:35.993432 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:35.993437 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:35.993440 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:35.993635 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:35.994127 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:35.994135 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:35.994451 (+ 316us) client_negotiation.cc:769] Sending connection context
0929 02:21:35.994688 (+ 237us) client_negotiation.cc:240] Negotiation successful
0929 02:21:35.995214 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":264,"threads_started":1}
I20260929 02:21:35.997650 22188 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:35.997931 22188 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:35.998586 22188 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:35.999879 21732 ts_manager.cc:194] Registered new tserver with Master: e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693)
I20260929 02:21:36.000626 21732 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:59279
I20260929 02:21:36.009017 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:21:36.025280 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.013679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54662 (local address 127.7.170.62:38141)
0929 02:21:36.013836 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.013842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.014224 (+ 382us) server_negotiation.cc:407] Connection header received
0929 02:21:36.014371 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.014374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.014449 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.014526 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:36.015770 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.016490 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.017736 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.018150 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.018240 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.020465 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:36.020469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:36.020472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:36.020502 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:36.022970 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.023565 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.023568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.023571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.023685 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.023993 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.023995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.023997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.024307 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:36.024511 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.024829 (+ 318us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.025051 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:21:36.029219 21732 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:21:36.035908 21732 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:21:36.071826 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.047927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35494 (local address 127.7.170.2:32909)
0929 02:21:36.064390 (+ 16463us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.064400 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.064433 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:21:36.064542 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.064550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.064783 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.064986 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:36.065824 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.067297 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.069718 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.070741 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.070861 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.071059 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.071341 (+ 282us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.071506 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16284,"thread_start_us":16159,"threads_started":1}
I20260929 02:21:36.071884 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.047682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32909 (local address 127.0.0.1:35494)
0929 02:21:36.064238 (+ 16556us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:36.064256 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:36.064395 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:36.065228 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:36.065232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:36.065255 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:36.065637 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.065648 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.067420 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.067425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.069599 (+ 2174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.069611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.070911 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.070917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.071118 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.071164 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:21:36.071273 (+ 109us) client_negotiation.cc:240] Negotiation successful
0929 02:21:36.071450 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16439,"mutex_wait_us":3,"thread_start_us":9781,"threads_started":1}
I20260929 02:21:36.072415 22202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.047514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33693 (local address 127.0.0.1:48202)
0929 02:21:36.057854 (+ 10340us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:36.057903 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:36.058744 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:36.064613 (+ 5869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:36.064621 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:36.064651 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:36.065174 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.065189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.067058 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.067062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.069409 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.069424 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.071311 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.071317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.071463 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.071493 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:21:36.071556 (+ 63us) client_negotiation.cc:240] Negotiation successful
0929 02:21:36.071691 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10081,"mutex_wait_us":1,"thread_start_us":16892,"threads_started":1}
I20260929 02:21:36.072994 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.049663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56450 (local address 127.7.170.1:45531)
0929 02:21:36.063787 (+ 14124us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.063797 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.064954 (+ 1157us) server_negotiation.cc:407] Connection header received
0929 02:21:36.065057 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.065064 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.065276 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.065438 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:36.066264 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.067610 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.070072 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.071449 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.071565 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.071731 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.072505 (+ 774us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.072661 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13934,"thread_start_us":13752,"threads_started":1}
I20260929 02:21:36.072667 22201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.049117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45531 (local address 127.0.0.1:56450)
0929 02:21:36.064673 (+ 15556us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:36.064688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:36.064804 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:36.065664 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:36.065668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:36.065696 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:36.066066 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.066078 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.067772 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.067776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.069912 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.069925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.071888 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.071894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.072043 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.072071 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:21:36.072143 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:21:36.072258 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15439,"thread_start_us":14837,"threads_started":1}
I20260929 02:21:36.074123 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.047797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.7.170.3:33693)
0929 02:21:36.063334 (+ 15537us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.063344 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.063382 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:21:36.063487 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.063496 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.063789 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.065397 (+ 1608us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:36.065665 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.066941 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.069590 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.070825 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.070952 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.071165 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.072706 (+ 1541us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.072873 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15342,"thread_start_us":15191,"threads_started":1}
I20260929 02:21:36.076596 21990 tablet_service.cc:1467] Processing CreateTablet for tablet 85b24d227af04252aa39d9bedc153d60 (DEFAULT_TABLE table=test-table [id=9f6c0e1806bd44eea33ef870413674af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:36.076596 21856 tablet_service.cc:1467] Processing CreateTablet for tablet 85b24d227af04252aa39d9bedc153d60 (DEFAULT_TABLE table=test-table [id=9f6c0e1806bd44eea33ef870413674af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:36.077922 21856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b24d227af04252aa39d9bedc153d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:36.077524 22123 tablet_service.cc:1467] Processing CreateTablet for tablet 85b24d227af04252aa39d9bedc153d60 (DEFAULT_TABLE table=test-table [id=9f6c0e1806bd44eea33ef870413674af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:36.078118 21990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b24d227af04252aa39d9bedc153d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:36.078784 22123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b24d227af04252aa39d9bedc153d60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:36.089584 22207 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Bootstrap starting.
I20260929 02:21:36.092444 22208 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Bootstrap starting.
I20260929 02:21:36.092845 22207 tablet_bootstrap.cc:654] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:36.094170 22207 log.cc:824] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:36.096086 22208 tablet_bootstrap.cc:654] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:36.097486 22208 log.cc:824] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:36.098733 22209 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Bootstrap starting.
I20260929 02:21:36.099972 22207 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: No bootstrap required, opened a new log
I20260929 02:21:36.100282 22207 ts_tablet_manager.cc:1397] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260929 02:21:36.100591 22208 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: No bootstrap required, opened a new log
I20260929 02:21:36.100844 22208 ts_tablet_manager.cc:1397] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260929 02:21:36.102475 22209 tablet_bootstrap.cc:654] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:36.103621 22209 log.cc:824] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:36.106386 22209 tablet_bootstrap.cc:492] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: No bootstrap required, opened a new log
I20260929 02:21:36.106613 22209 ts_tablet_manager.cc:1397] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:21:36.111418 22207 raft_consensus.cc:348] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.111418 22208 raft_consensus.cc:348] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.111876 22208 raft_consensus.cc:374] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:36.111881 22207 raft_consensus.cc:374] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:36.111975 22207 raft_consensus.cc:729] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37d1d4296f284fe6b440fbe3062779bc, State: Initialized, Role: FOLLOWER
I20260929 02:21:36.111975 22208 raft_consensus.cc:729] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9858cd94f345d3b54a4cccb3f441b9, State: Initialized, Role: FOLLOWER
I20260929 02:21:36.112617 22208 consensus_queue.cc:260] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.112615 22207 consensus_queue.cc:260] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.114384 22209 raft_consensus.cc:348] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.114797 22209 raft_consensus.cc:374] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:36.114806 22208 ts_tablet_manager.cc:1428] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260929 02:21:36.114897 22209 raft_consensus.cc:729] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aed72b1efc455e80bc50666a159a81, State: Initialized, Role: FOLLOWER
I20260929 02:21:36.115121 22055 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:36.115561 22209 consensus_queue.cc:260] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.116482 22207 ts_tablet_manager.cc:1428] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260929 02:21:36.117333 22188 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:36.117920 22209 ts_tablet_manager.cc:1428] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260929 02:21:36.201869 21922 tablet.cc:2369] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:21:36.201910 22056 tablet.cc:2369] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:21:36.227576 22189 tablet.cc:2369] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:21:36.327414 22213 raft_consensus.cc:482] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:21:36.327715 22213 raft_consensus.cc:504] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.329493 22213 leader_election.cc:288] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693), 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909)
I20260929 02:21:36.336217 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.330099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:57891 (local address 127.7.170.2:32909)
0929 02:21:36.330310 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.330329 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.331023 (+ 694us) server_negotiation.cc:407] Connection header received
0929 02:21:36.331165 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.331169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.331227 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.331315 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:36.331927 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.332647 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.334561 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.335306 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.335427 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.335530 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.335832 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.335965 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:21:36.336236 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.330013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:40845 (local address 127.7.170.3:33693)
0929 02:21:36.330203 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.330209 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.330934 (+ 725us) server_negotiation.cc:407] Connection header received
0929 02:21:36.331106 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.331112 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.331185 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.331251 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:36.331900 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.332626 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.334547 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.335316 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.335432 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.335530 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.335890 (+ 360us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.336016 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:21:36.336336 22217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.330020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32909 (local address 127.7.170.1:57891)
0929 02:21:36.330893 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:36.330916 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:36.331045 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:36.331347 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:36.331352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:36.331375 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:36.331759 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.331768 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.332771 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.332774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.334406 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.334418 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.335530 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.335537 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.335677 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.335704 (+ 27us) client_negotiation.cc:769] Sending connection context
0929 02:21:36.335776 (+ 72us) client_negotiation.cc:240] Negotiation successful
0929 02:21:36.335894 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":94,"thread_start_us":481,"threads_started":1}
I20260929 02:21:36.336819 22216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.329995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33693 (local address 127.7.170.1:40845)
0929 02:21:36.330818 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:36.330833 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:36.330959 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:36.331303 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:36.331307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:36.331331 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:36.331743 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.331753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.332772 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.332777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.334390 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:36.334402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.335591 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.335596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:36.335725 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.335747 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:21:36.335835 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:21:36.335957 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":10,"thread_start_us":443,"threads_started":1}
I20260929 02:21:36.337512 22010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b24d227af04252aa39d9bedc153d60" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" is_pre_election: true
I20260929 02:21:36.337765 22143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b24d227af04252aa39d9bedc153d60" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9aed72b1efc455e80bc50666a159a81" is_pre_election: true
I20260929 02:21:36.338101 22010 raft_consensus.cc:2454] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 0.
I20260929 02:21:36.338259 22143 raft_consensus.cc:2454] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 0.
I20260929 02:21:36.338761 21810 leader_election.cc:302] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37d1d4296f284fe6b440fbe3062779bc, 9a9858cd94f345d3b54a4cccb3f441b9; no voters:
I20260929 02:21:36.339155 22213 raft_consensus.cc:2781] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:21:36.339277 22213 raft_consensus.cc:482] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:21:36.339376 22213 raft_consensus.cc:3037] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:36.341588 22213 raft_consensus.cc:504] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.342166 22213 leader_election.cc:288] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 election: Requested vote from peers e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693), 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909)
I20260929 02:21:36.342636 22143 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b24d227af04252aa39d9bedc153d60" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9aed72b1efc455e80bc50666a159a81"
I20260929 02:21:36.342643 22010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b24d227af04252aa39d9bedc153d60" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
I20260929 02:21:36.342811 22010 raft_consensus.cc:3037] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:36.342854 22143 raft_consensus.cc:3037] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:36.345031 22010 raft_consensus.cc:2454] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 1.
I20260929 02:21:36.345297 22143 raft_consensus.cc:2454] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 1.
I20260929 02:21:36.345399 21810 leader_election.cc:302] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37d1d4296f284fe6b440fbe3062779bc, 9a9858cd94f345d3b54a4cccb3f441b9; no voters:
I20260929 02:21:36.345697 22213 raft_consensus.cc:2781] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:36.346139 22213 raft_consensus.cc:686] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 1 LEADER]: Becoming Leader. State: Replica: 37d1d4296f284fe6b440fbe3062779bc, State: Running, Role: LEADER
I20260929 02:21:36.346606 22213 consensus_queue.cc:237] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:36.350747 21731 catalog_manager.cc:5495] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1). New cstate: current_term: 1 leader_uuid: "37d1d4296f284fe6b440fbe3062779bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } health_report { overall_health: HEALTHY } } }
I20260929 02:21:36.368305 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.363050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.7.170.1:45531)
0929 02:21:36.363239 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.363244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.363264 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:21:36.363451 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.363455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.363550 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.363666 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:21:36.364036 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.364765 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.366232 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.366645 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.366744 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.367230 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:21:36.367747 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:21:36.367852 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.367982 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.368051 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:21:36.382781 22143 raft_consensus.cc:1261] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Refusing update from remote peer 37d1d4296f284fe6b440fbe3062779bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:36.382802 22010 raft_consensus.cc:1261] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Refusing update from remote peer 37d1d4296f284fe6b440fbe3062779bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:36.383846 22218 consensus_queue.cc:1035] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:36.384217 22213 consensus_queue.cc:1035] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:36.410989 21732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:36.411294 21732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54662:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:36.414942 21732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Sending DeleteTablet for 3 replicas of tablet 85b24d227af04252aa39d9bedc153d60
I20260929 02:21:36.416177 22123 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b24d227af04252aa39d9bedc153d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:48202
I20260929 02:21:36.416177 21990 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b24d227af04252aa39d9bedc153d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:35494
I20260929 02:21:36.416468 21856 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b24d227af04252aa39d9bedc153d60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:36 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:56450
I20260929 02:21:36.417624 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21793
I20260929 02:21:36.427518 22233 tablet_replica.cc:330] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: stopping tablet replica
I20260929 02:21:36.428032 22233 raft_consensus.cc:2229] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:21:36.428750 22233 raft_consensus.cc:2258] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:36.429720 22232 tablet_replica.cc:330] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: stopping tablet replica
I20260929 02:21:36.430160 22232 raft_consensus.cc:2229] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:36.430677 22232 raft_consensus.cc:2258] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:36.432441 22233 ts_tablet_manager.cc:1905] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:36.440507 22232 ts_tablet_manager.cc:1905] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Deleting tablet data with delete state TABLET_DATA_DELETED
Sep 29 02:21:36 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:21:36.447579 22232 ts_tablet_manager.cc:1918] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:36.447752 22232 log.cc:1196] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/85b24d227af04252aa39d9bedc153d60
I20260929 02:21:36.448235 22232 ts_tablet_manager.cc:1939] T 85b24d227af04252aa39d9bedc153d60 P e9aed72b1efc455e80bc50666a159a81: Deleting consensus metadata
I20260929 02:21:36.449430 22231 tablet_replica.cc:330] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: stopping tablet replica
I20260929 02:21:36.449847 22231 raft_consensus.cc:2229] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:36.450412 22231 raft_consensus.cc:2258] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:36.451972 22233 ts_tablet_manager.cc:1918] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:36.452179 22233 log.cc:1196] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/85b24d227af04252aa39d9bedc153d60
I20260929 02:21:36.452827 22233 ts_tablet_manager.cc:1939] T 85b24d227af04252aa39d9bedc153d60 P 37d1d4296f284fe6b440fbe3062779bc: Deleting consensus metadata
I20260929 02:21:36.455152 21716 catalog_manager.cc:4841] TS 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1:45531): tablet 85b24d227af04252aa39d9bedc153d60 (table test-table [id=9f6c0e1806bd44eea33ef870413674af]) successfully deleted
I20260929 02:21:36.456089 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.418850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.7.170.1:45531)
0929 02:21:36.419063 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.419069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.433206 (+ 14137us) server_negotiation.cc:407] Connection header received
0929 02:21:36.433513 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.433520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.433627 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.433706 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:36.435898 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.437314 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.439784 (+ 2470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.440524 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.440640 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.446919 (+ 6279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:36.446940 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:36.446963 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:36.447017 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:36.451988 (+ 4971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.453104 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.453114 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.453117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.453318 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.453786 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.453790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.453793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.454275 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:36.454651 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.455519 (+ 868us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.455754 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:21:36.456651 21719 catalog_manager.cc:4841] TS e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693): tablet 85b24d227af04252aa39d9bedc153d60 (table test-table [id=9f6c0e1806bd44eea33ef870413674af]) successfully deleted
I20260929 02:21:36.456698 22231 ts_tablet_manager.cc:1905] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:36.457494 21916 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:36.463207 22231 ts_tablet_manager.cc:1918] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:36.463598 22231 log.cc:1196] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/85b24d227af04252aa39d9bedc153d60
I20260929 02:21:36.464136 22231 ts_tablet_manager.cc:1939] T 85b24d227af04252aa39d9bedc153d60 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting consensus metadata
I20260929 02:21:36.465823 21717 catalog_manager.cc:4841] TS 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909): tablet 85b24d227af04252aa39d9bedc153d60 (table test-table [id=9f6c0e1806bd44eea33ef870413674af]) successfully deleted
I20260929 02:21:36.560715 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21793
I20260929 02:21:36.576489 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21927
Sep 29 02:21:36 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:21:36.590979 22204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.577319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.7.170.2:32909)
0929 02:21:36.577482 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.577486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.577507 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:21:36.577617 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.577621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.577702 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.577783 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:36.578960 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.579709 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.581094 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.581500 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.581598 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.585649 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:36.585665 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:36.585681 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:36.585720 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:36.588323 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.589133 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.589142 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.589144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.589277 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.589628 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.589631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.589633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.590029 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:36.590269 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.590571 (+ 302us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.590730 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:21:36.591636 22050 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:36.680974 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21927
I20260929 02:21:36.694697 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22060
Sep 29 02:21:36 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:21:36.709350 22203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:36.695373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48218 (local address 127.7.170.3:33693)
0929 02:21:36.695583 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:36.695589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:36.695610 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:21:36.695758 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:36.695762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:36.695828 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:36.695912 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:36.697247 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.697949 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.699269 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:36.699657 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:36.699765 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:36.703920 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:36.703936 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:36.703952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:36.703999 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:36.706630 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.707285 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.707293 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.707296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.707457 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:36.707808 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:36.707812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:36.707814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:36.708160 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:36.708435 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:36.708752 (+ 317us) server_negotiation.cc:299] Negotiation successful
0929 02:21:36.709033 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:21:36.709946 22183 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:36.801561 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22060
I20260929 02:21:36.815594 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 21700
I20260929 02:21:36.816625 21762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:36.927713 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 21700
I20260929 02:21:36.945729 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--webserver_interface=127.7.170.62
--webserver_port=36381
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:37.260443 22239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:37.260744 22239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:37.260882 22239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:37.268033 22239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:37.268159 22239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:37.268237 22239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:37.268314 22239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:37.268371 22239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:37.278091 22239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=36381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:37.279335 22239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:37.280581 22239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:37.302857 22245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:37.302918 22248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:37.302886 22246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:37.303740 22239 server_base.cc:1034] running on GCE node
I20260929 02:21:37.304656 22239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:37.306521 22239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:37.307746 22239 hybrid_clock.cc:648] HybridClock initialized: now 1790648497307677 us; error 59 us; skew 500 ppm
I20260929 02:21:37.308225 22239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:37 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648497, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:37.311841 22239 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:37.313573 22239 webserver.cc:460] Webserver started at http://127.7.170.62:36381/ using document root <none> and password file <none>
I20260929 02:21:37.314360 22239 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:37.314496 22239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:37.320533 22239 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260929 02:21:37.323599 22255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:37.325120 22239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:37.325292 22239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e742251329a84c839c44af920ccf4baf"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "d2c1ead3bc5a37eede10aa11bdede8cb"
server_key_iv: "0dbc23d4159f394968f6f260ecc744fc"
server_key_version: "encryptionkey@0"
I20260929 02:21:37.326457 22239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:37.338953 22239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:37.343083 22239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:37.343333 22239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:37.361634 22239 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38141
I20260929 02:21:37.361657 22306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38141 every 8 connection(s)
I20260929 02:21:37.363224 22239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:37.366950 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22239
I20260929 02:21:37.367722 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:45531
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=37067
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:37.371037 22307 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:21:37.374567 22307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap starting.
I20260929 02:21:37.402355 22307 log.cc:824] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:37.418597 22307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:21:37.419255 22307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap complete.
I20260929 02:21:37.433324 22307 raft_consensus.cc:348] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:37.434954 22307 raft_consensus.cc:729] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e742251329a84c839c44af920ccf4baf, State: Initialized, Role: FOLLOWER
I20260929 02:21:37.435894 22307 consensus_queue.cc:260] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:37.436249 22307 raft_consensus.cc:388] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:37.436405 22307 raft_consensus.cc:482] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:37.436620 22307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:21:37.442742 22307 raft_consensus.cc:504] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:37.443442 22307 leader_election.cc:302] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e742251329a84c839c44af920ccf4baf; no voters:
I20260929 02:21:37.444118 22307 leader_election.cc:288] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:21:37.444249 22311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:21:37.445345 22311 raft_consensus.cc:686] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 LEADER]: Becoming Leader. State: Replica: e742251329a84c839c44af920ccf4baf, State: Running, Role: LEADER
I20260929 02:21:37.445887 22311 consensus_queue.cc:237] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:37.446384 22307 sys_catalog.cc:564] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:37.448194 22312 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:37.448187 22313 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e742251329a84c839c44af920ccf4baf. Latest consensus state: current_term: 2 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:37.448611 22312 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:37.448608 22313 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:37.449463 22317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:37.464018 22317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:37.465569 22317 catalog_manager.cc:1259] Loaded cluster ID: 9695d43113694c0897d04e66d5ca8b25
I20260929 02:21:37.465737 22317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:37.469059 22317 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260929 02:21:37.487246 22317 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260929 02:21:37.507524 22317 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:37.509441 22317 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Loaded TSK: 0
I20260929 02:21:37.511061 22317 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:21:37.745867 22309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:37.746153 22309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:37.746299 22309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:37.753731 22309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:37.753865 22309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:37.754045 22309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:37.763739 22309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:45531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=37067
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:37.765102 22309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:37.766424 22309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:37.781157 22334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:37.781124 22335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:37.781809 22337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:37.782241 22309 server_base.cc:1034] running on GCE node
I20260929 02:21:37.782984 22309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:37.784180 22309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:37.785416 22309 hybrid_clock.cc:648] HybridClock initialized: now 1790648497785352 us; error 51 us; skew 500 ppm
I20260929 02:21:37.785809 22309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:37 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648497, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:37.789721 22309 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:37.791508 22309 webserver.cc:460] Webserver started at http://127.7.170.1:37067/ using document root <none> and password file <none>
I20260929 02:21:37.792400 22309 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:37.792542 22309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:37.799530 22309 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.001s
I20260929 02:21:37.802838 22344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:37.804559 22309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:37.804744 22309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "37d1d4296f284fe6b440fbe3062779bc"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "a7ddb116109ce89e1652964e10c6ac56"
server_key_iv: "60686093683b0e84b43f8e54180732cc"
server_key_version: "encryptionkey@0"
I20260929 02:21:37.806147 22309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:37.821704 22309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:37.826892 22309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:37.827172 22309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:37.828330 22309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:37.830020 22309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:37.830121 22309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:37.830219 22309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:37.830289 22309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:37.860481 22309 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:45531
I20260929 02:21:37.860551 22456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:45531 every 8 connection(s)
I20260929 02:21:37.861971 22309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:37.865856 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22309
I20260929 02:21:37.866539 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:32909
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=38215
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:37 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648497, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:37.885231 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:37.864687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:52467 (local address 127.7.170.62:38141)
0929 02:21:37.865325 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:37.865341 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:37.865908 (+ 567us) server_negotiation.cc:407] Connection header received
0929 02:21:37.868080 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:37.868093 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:37.868424 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:37.868758 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:37.870292 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:37.871587 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:37.873185 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:37.873856 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:37.874003 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:37.877923 (+ 3920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:37.877931 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:37.877946 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:37.877984 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:37.880763 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:37.881649 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:37.881655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:37.881661 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:37.881826 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:37.882267 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:37.882269 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:37.882271 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:37.882739 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:37.882923 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:37.883855 (+ 932us) server_negotiation.cc:299] Negotiation successful
0929 02:21:37.884212 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":260,"threads_started":1}
I20260929 02:21:37.885488 22460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:37.864836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:52467)
0929 02:21:37.865726 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:37.865761 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:37.867539 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:37.868921 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:37.868956 (+ 35us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:37.869554 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:37.870083 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:37.870100 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:37.871741 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:37.871745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:37.873043 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:37.873058 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:37.874079 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:37.874086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:37.874345 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:37.875029 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:37.875055 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:37.877620 (+ 2565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:37.880905 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:37.880913 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:37.880949 (+ 36us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:37.881512 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:37.881930 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:37.881934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:37.881938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:37.882176 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:37.882876 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:37.882883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:37.883300 (+ 417us) client_negotiation.cc:769] Sending connection context
0929 02:21:37.883626 (+ 326us) client_negotiation.cc:240] Negotiation successful
0929 02:21:37.884187 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":442,"threads_started":1}
I20260929 02:21:37.888191 22457 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:37.888484 22457 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:37.889184 22457 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:37.891160 22272 ts_manager.cc:194] Registered new tserver with Master: 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1:45531)
I20260929 02:21:37.892612 22272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:52467
W20260929 02:21:38.183614 22461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:38.183902 22461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:38.184039 22461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:38.191253 22461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:38.191376 22461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:38.191529 22461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:38.201138 22461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:32909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=38215
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:38.202385 22461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:38.203668 22461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:38.216353 22466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:38.216435 22469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:38.216447 22467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:38.217374 22461 server_base.cc:1034] running on GCE node
I20260929 02:21:38.217963 22461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:38.218905 22461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:38.220117 22461 hybrid_clock.cc:648] HybridClock initialized: now 1790648498220062 us; error 49 us; skew 500 ppm
I20260929 02:21:38.220402 22461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:38 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:38.223512 22461 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:38.225014 22461 webserver.cc:460] Webserver started at http://127.7.170.2:38215/ using document root <none> and password file <none>
I20260929 02:21:38.225698 22461 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:38.225827 22461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:38.231249 22461 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:21:38.234022 22476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.235466 22461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:38.235617 22461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "74306fd30fa03ca3871eec5219c4ae10"
server_key_iv: "d12d002715f5d70e8ec38cc3f5307629"
server_key_version: "encryptionkey@0"
I20260929 02:21:38.236574 22461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:38.268419 22461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:38.272815 22461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:38.273092 22461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:38.274063 22461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:38.275353 22461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:38.275451 22461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.275554 22461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:38.275625 22461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.305496 22461 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:32909
I20260929 02:21:38.305562 22588 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:32909 every 8 connection(s)
I20260929 02:21:38.306907 22461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:38.310482 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22461
I20260929 02:21:38.311216 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:33693
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=46837
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:21:38 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:38.326527 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.309294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:42009 (local address 127.7.170.62:38141)
0929 02:21:38.309490 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.309496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.310704 (+ 1208us) server_negotiation.cc:407] Connection header received
0929 02:21:38.312867 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.312872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.312988 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.313080 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:38.314647 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.315384 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.316971 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.317400 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.317513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.321072 (+ 3559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:38.321078 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:38.321081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:38.321117 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:38.323505 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.324224 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.324229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.324231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.324358 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.324764 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.324768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.324770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.325088 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:38.325232 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.325938 (+ 706us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.326181 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:21:38.327521 22591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.309812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:42009)
0929 02:21:38.310536 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.310588 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.312588 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.313344 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.313355 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.313874 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:38.314470 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.314489 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.315535 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.315541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.316794 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.316807 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.317622 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.317629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.317865 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.318496 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:38.318519 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:38.320827 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:38.323660 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:38.323669 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:38.323688 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:38.324107 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:38.324463 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:38.324468 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:38.324472 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:38.324658 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:38.325217 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:38.325226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:38.325572 (+ 346us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.325827 (+ 255us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.326383 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":287,"threads_started":1}
I20260929 02:21:38.329133 22589 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:38.329499 22589 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:38.330467 22589 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:38.332180 22272 ts_manager.cc:194] Registered new tserver with Master: 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909)
I20260929 02:21:38.333341 22272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:42009
W20260929 02:21:38.628966 22592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:38.629256 22592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:38.629359 22592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:38.636610 22592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:38.636727 22592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:38.636843 22592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:38.647003 22592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=46837
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:38.648298 22592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:38.649586 22592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:38.661967 22597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:38.662071 22600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:38.662091 22598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:38.663136 22592 server_base.cc:1034] running on GCE node
I20260929 02:21:38.663712 22592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:38.664670 22592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:38.665860 22592 hybrid_clock.cc:648] HybridClock initialized: now 1790648498665796 us; error 53 us; skew 500 ppm
I20260929 02:21:38.666152 22592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:38 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:38.669186 22592 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:38.670599 22592 webserver.cc:460] Webserver started at http://127.7.170.3:46837/ using document root <none> and password file <none>
I20260929 02:21:38.671265 22592 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:38.671386 22592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:38.676684 22592 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:38.679338 22607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.680678 22592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:38.680835 22592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e9aed72b1efc455e80bc50666a159a81"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "4dff5e9a17a1c53a62bbdc66704a134c"
server_key_iv: "2edb6ffe41e873b8e3e3e405b0a744a1"
server_key_version: "encryptionkey@0"
I20260929 02:21:38.682072 22592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:38.702193 22592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:38.706146 22592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:38.706362 22592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:38.707290 22592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:38.708554 22592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:38.708649 22592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.708729 22592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:38.708781 22592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:38.741428 22592 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33693
I20260929 02:21:38.741459 22719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33693 every 8 connection(s)
I20260929 02:21:38.742856 22592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:21:38.743846 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22592
Sep 29 02:21:38 dist-test-slave-8cs2 krb5kdc[21687](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648498, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:38.759876 22459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.744902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54664 (local address 127.7.170.62:38141)
0929 02:21:38.745148 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.745154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.745521 (+ 367us) server_negotiation.cc:407] Connection header received
0929 02:21:38.745721 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.745728 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.745826 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.745933 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:38.747870 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.749001 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.750595 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.751210 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.751325 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.754018 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:38.754022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:38.754025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:38.754057 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:38.757001 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.757768 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.757772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.757774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.757922 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.758308 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.758311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.758313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.758648 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:38.758950 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.759274 (+ 324us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.759556 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260929 02:21:38.764226 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.745635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:47647 (local address 127.7.170.62:38141)
0929 02:21:38.748050 (+ 2415us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.748054 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.748068 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:21:38.749563 (+ 1495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.749569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.749644 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.749757 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:38.751278 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.752336 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.753881 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.754410 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.754494 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.758466 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:38.758472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:38.758476 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:38.758506 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:38.761571 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.762236 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.762241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.762244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.762381 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:38.762793 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:38.762796 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:38.762798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:38.763049 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:38.763150 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.763775 (+ 625us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.763955 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2328,"thread_start_us":2221,"threads_started":1}
I20260929 02:21:38.765256 22724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.746291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:47647)
0929 02:21:38.747079 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.747123 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.749319 (+ 2196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.749919 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.749927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.750405 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:38.751090 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.751114 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.752486 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.752490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.753745 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.753757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.754614 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.754619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.754882 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.755387 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:38.755430 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:38.758230 (+ 2800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:38.761688 (+ 3458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:38.761695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:38.761710 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:38.762141 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:38.762499 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:38.762504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:38.762508 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:38.762709 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:38.763139 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:38.763145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:38.763476 (+ 331us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.763709 (+ 233us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.764111 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":309,"threads_started":1}
I20260929 02:21:38.766448 22720 heartbeater.cc:347] Connected to a master server at 127.7.170.62:38141
I20260929 02:21:38.766734 22720 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:38.767421 22720 heartbeater.cc:511] Master 127.7.170.62:38141 requested a full tablet report, sending...
I20260929 02:21:38.768692 22272 ts_manager.cc:194] Registered new tserver with Master: e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693)
I20260929 02:21:38.769501 22272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:47647
I20260929 02:21:38.774415 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:21:38.780364 22723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.775201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54672 (local address 127.7.170.62:38141)
0929 02:21:38.775371 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.775376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.775801 (+ 425us) server_negotiation.cc:407] Connection header received
0929 02:21:38.775959 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.775962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.776090 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.776202 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:21:38.776555 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.777239 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.778578 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.778973 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.779059 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.779468 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:21:38.779953 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:21:38.780061 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.780123 (+ 62us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.780185 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:21:38.781234 22272 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:54672:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:21:38.788404 22272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:21:38.821749 22733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.800582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45531 (local address 127.0.0.1:56486)
0929 02:21:38.813663 (+ 13081us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.813682 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.813817 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.814346 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.814354 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.814383 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.814775 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.814787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.816667 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.816673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.818753 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.818763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.820643 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.820649 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.820897 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.820977 (+ 80us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.821115 (+ 138us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.821291 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12942,"mutex_wait_us":40,"thread_start_us":12776,"threads_started":1}
I20260929 02:21:38.822366 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.800717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.7.170.1:45531)
0929 02:21:38.813105 (+ 12388us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.813115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.813829 (+ 714us) server_negotiation.cc:407] Connection header received
0929 02:21:38.813935 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.813940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.814156 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.814305 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.814966 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.816522 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.818975 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.820429 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.820579 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.820798 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.821992 (+ 1194us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.822130 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12183,"thread_start_us":13724,"threads_started":1}
I20260929 02:21:38.822919 22730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.800117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33693 (local address 127.0.0.1:48228)
0929 02:21:38.813599 (+ 13482us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.813616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.813714 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.815790 (+ 2076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.815795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.815821 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.816178 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.816189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.818116 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.818121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.820179 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.820194 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.822020 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.822031 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.822206 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.822238 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.822326 (+ 88us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.822451 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13352,"thread_start_us":10838,"threads_started":1}
I20260929 02:21:38.823745 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.800714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48228 (local address 127.7.170.3:33693)
0929 02:21:38.815029 (+ 14315us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.815039 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.815071 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:21:38.815161 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.815169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.815415 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.816362 (+ 947us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.816611 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.818002 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.820314 (+ 2312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.821617 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.821739 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.821970 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.822389 (+ 419us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.822550 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14106,"thread_start_us":14008,"threads_started":1}
I20260929 02:21:38.824976 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.799845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32909 (local address 127.0.0.1:35506)
0929 02:21:38.811974 (+ 12129us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.812025 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.812225 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.817856 (+ 5631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.817861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.817887 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.818320 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.818332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.820129 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.820135 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.821942 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.821954 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.824267 (+ 2313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.824271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.824389 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.824409 (+ 20us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.824487 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.824608 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11249,"mutex_wait_us":1,"thread_start_us":13445,"threads_started":1}
I20260929 02:21:38.825071 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.800327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35506 (local address 127.7.170.2:32909)
0929 02:21:38.817317 (+ 16990us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.817324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.817355 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:21:38.817451 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.817457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.817662 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.817835 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.818497 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.819994 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.822124 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.823793 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.823912 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.824143 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.824557 (+ 414us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.824723 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16820,"thread_start_us":16707,"threads_started":1}
I20260929 02:21:38.826789 22392 tablet_service.cc:1467] Processing CreateTablet for tablet a17c1ac4ffba4a48a57c932736e11308 (DEFAULT_TABLE table=test-table [id=d057f512ad3b49c68f371885ec68a36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:38.827284 22655 tablet_service.cc:1467] Processing CreateTablet for tablet a17c1ac4ffba4a48a57c932736e11308 (DEFAULT_TABLE table=test-table [id=d057f512ad3b49c68f371885ec68a36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:38.828275 22392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17c1ac4ffba4a48a57c932736e11308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:38.827937 22524 tablet_service.cc:1467] Processing CreateTablet for tablet a17c1ac4ffba4a48a57c932736e11308 (DEFAULT_TABLE table=test-table [id=d057f512ad3b49c68f371885ec68a36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:21:38.828588 22655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17c1ac4ffba4a48a57c932736e11308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:38.829303 22524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a17c1ac4ffba4a48a57c932736e11308. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:38.840698 22736 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Bootstrap starting.
I20260929 02:21:38.844218 22738 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Bootstrap starting.
I20260929 02:21:38.844776 22736 tablet_bootstrap.cc:654] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:38.845793 22737 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Bootstrap starting.
I20260929 02:21:38.846349 22736 log.cc:824] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:38.847708 22738 tablet_bootstrap.cc:654] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:38.848865 22737 tablet_bootstrap.cc:654] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Neither blocks nor log segments found. Creating new log.
I20260929 02:21:38.848897 22738 log.cc:824] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:38.849736 22736 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: No bootstrap required, opened a new log
I20260929 02:21:38.849922 22737 log.cc:824] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:38.849972 22736 ts_tablet_manager.cc:1397] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260929 02:21:38.852339 22738 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: No bootstrap required, opened a new log
I20260929 02:21:38.852499 22737 tablet_bootstrap.cc:492] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: No bootstrap required, opened a new log
I20260929 02:21:38.852604 22738 ts_tablet_manager.cc:1397] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:21:38.852732 22737 ts_tablet_manager.cc:1397] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260929 02:21:38.859889 22737 raft_consensus.cc:348] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.860435 22737 raft_consensus.cc:374] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:38.860526 22737 raft_consensus.cc:729] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9aed72b1efc455e80bc50666a159a81, State: Initialized, Role: FOLLOWER
I20260929 02:21:38.860564 22736 raft_consensus.cc:348] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.860986 22736 raft_consensus.cc:374] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:38.861075 22736 raft_consensus.cc:729] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a9858cd94f345d3b54a4cccb3f441b9, State: Initialized, Role: FOLLOWER
I20260929 02:21:38.861156 22737 consensus_queue.cc:260] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.861729 22736 consensus_queue.cc:260] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.863021 22738 raft_consensus.cc:348] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.863466 22738 raft_consensus.cc:374] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:38.863547 22738 raft_consensus.cc:729] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37d1d4296f284fe6b440fbe3062779bc, State: Initialized, Role: FOLLOWER
I20260929 02:21:38.864319 22738 consensus_queue.cc:260] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.870132 22736 ts_tablet_manager.cc:1428] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Time spent starting tablet: real 0.020s user 0.008s sys 0.011s
I20260929 02:21:38.870563 22589 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:38.878357 22457 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:38.879083 22738 ts_tablet_manager.cc:1428] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Time spent starting tablet: real 0.026s user 0.014s sys 0.011s
I20260929 02:21:38.880726 22737 ts_tablet_manager.cc:1428] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Time spent starting tablet: real 0.028s user 0.020s sys 0.008s
I20260929 02:21:38.880949 22720 heartbeater.cc:503] Master 127.7.170.62:38141 was elected leader, sending a full tablet report...
I20260929 02:21:38.966689 22744 raft_consensus.cc:482] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:21:38.966997 22744 raft_consensus.cc:504] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.968652 22744 leader_election.cc:288] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909), e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693)
I20260929 02:21:38.972139 22743 raft_consensus.cc:482] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:21:38.972461 22743 raft_consensus.cc:504] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.976064 22743 leader_election.cc:288] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693), 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1:45531)
I20260929 02:21:38.977489 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.969091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:54959 (local address 127.7.170.2:32909)
0929 02:21:38.969299 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.969305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.969995 (+ 690us) server_negotiation.cc:407] Connection header received
0929 02:21:38.970231 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.970235 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.970297 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.970373 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.970978 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.972089 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.974661 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.976011 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.976113 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.976214 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.977095 (+ 881us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.977227 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:21:38.977584 22746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.969195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:32909 (local address 127.7.170.1:54959)
0929 02:21:38.969880 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.969902 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.970073 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.970414 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.970418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.970439 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.970796 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.970808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.972258 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.972263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.974469 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.974484 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.976693 (+ 2209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.976699 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.976894 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.976969 (+ 75us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.977060 (+ 91us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.977198 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":58,"spinlock_wait_cycles":28032,"thread_start_us":395,"threads_started":1}
I20260929 02:21:38.978544 22544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" is_pre_election: true
I20260929 02:21:38.978998 22544 raft_consensus.cc:2454] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 0.
I20260929 02:21:38.979282 22745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.969195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33693 (local address 127.7.170.1:39715)
0929 02:21:38.970269 (+ 1074us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.970283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.970405 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.970699 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.970707 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.970734 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.971101 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.971109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.972504 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.972508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.974129 (+ 1621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.974142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.977837 (+ 3695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.977843 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.978017 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.978713 (+ 696us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.978795 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.978923 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":665,"thread_start_us":750,"threads_started":1}
I20260929 02:21:38.979513 22735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.969199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:39715 (local address 127.7.170.3:33693)
0929 02:21:38.969376 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.969380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.970369 (+ 989us) server_negotiation.cc:407] Connection header received
0929 02:21:38.970533 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.970536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.970591 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.970677 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.971307 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.972358 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.974302 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.975448 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.975568 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.975671 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.979133 (+ 3462us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.979265 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:21:38.979775 22346 leader_election.cc:302] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37d1d4296f284fe6b440fbe3062779bc, 9a9858cd94f345d3b54a4cccb3f441b9; no voters:
I20260929 02:21:38.980221 22744 raft_consensus.cc:2781] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:21:38.980355 22744 raft_consensus.cc:482] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:21:38.980444 22744 raft_consensus.cc:3037] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:38.980517 22675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9aed72b1efc455e80bc50666a159a81" is_pre_election: true
I20260929 02:21:38.981489 22675 raft_consensus.cc:2454] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 0.
I20260929 02:21:38.983494 22744 raft_consensus.cc:504] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.984077 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.975102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:34991 (local address 127.7.170.1:45531)
0929 02:21:38.975261 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.975266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.976437 (+ 1171us) server_negotiation.cc:407] Connection header received
0929 02:21:38.976690 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.976696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.976772 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.976830 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.978019 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.978979 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.981926 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.983223 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.983351 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.983433 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.983708 (+ 275us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.983827 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":55}
I20260929 02:21:38.984169 22747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.974754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:45531 (local address 127.7.170.2:34991)
0929 02:21:38.976342 (+ 1588us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.976366 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.976544 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.977081 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.977086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.977115 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.977530 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.977849 (+ 319us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.979647 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.979652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.981734 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.981752 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.983410 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.983416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.983569 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.983591 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.983667 (+ 76us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.983787 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"mutex_wait_us":1,"thread_start_us":1077,"threads_started":1}
I20260929 02:21:38.984608 22544 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
I20260929 02:21:38.984754 22744 leader_election.cc:288] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 election: Requested vote from peers 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909), e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693)
I20260929 02:21:38.984814 22544 raft_consensus.cc:3037] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:38.985078 22675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "37d1d4296f284fe6b440fbe3062779bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9aed72b1efc455e80bc50666a159a81"
I20260929 02:21:38.985278 22675 raft_consensus.cc:3037] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:38.985263 22412 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "37d1d4296f284fe6b440fbe3062779bc" is_pre_election: true
I20260929 02:21:38.985745 22412 raft_consensus.cc:2379] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a9858cd94f345d3b54a4cccb3f441b9 in current term 1: Already voted for candidate 37d1d4296f284fe6b440fbe3062779bc in this term.
I20260929 02:21:38.987491 22544 raft_consensus.cc:2454] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 1.
I20260929 02:21:38.987561 22748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.974825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33693 (local address 127.7.170.2:53589)
0929 02:21:38.978423 (+ 3598us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:38.978436 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:38.978553 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:38.979037 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:38.979043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:38.979067 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:21:38.979481 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.979495 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.981160 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.981165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.983135 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:38.983151 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.986812 (+ 3661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.986819 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:38.986987 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.987020 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:21:38.987106 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:21:38.987234 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3496,"thread_start_us":7425,"threads_started":1}
I20260929 02:21:38.987846 22346 leader_election.cc:302] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 37d1d4296f284fe6b440fbe3062779bc, 9a9858cd94f345d3b54a4cccb3f441b9; no voters:
I20260929 02:21:38.988081 22744 raft_consensus.cc:2781] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:38.988286 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:38.975280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:53589 (local address 127.7.170.3:33693)
0929 02:21:38.976677 (+ 1397us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:38.976683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:38.978516 (+ 1833us) server_negotiation.cc:407] Connection header received
0929 02:21:38.978665 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:38.978670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:38.978738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:38.978827 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:21:38.979674 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.980991 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.983311 (+ 2320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:38.986428 (+ 3117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:38.986547 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:38.986657 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:38.987806 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0929 02:21:38.987949 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1262,"thread_start_us":1083,"threads_started":1}
I20260929 02:21:38.988451 22675 raft_consensus.cc:2454] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 37d1d4296f284fe6b440fbe3062779bc in term 1.
I20260929 02:21:38.988842 22674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a17c1ac4ffba4a48a57c932736e11308" candidate_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9aed72b1efc455e80bc50666a159a81" is_pre_election: true
I20260929 02:21:38.989171 22744 raft_consensus.cc:686] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 LEADER]: Becoming Leader. State: Replica: 37d1d4296f284fe6b440fbe3062779bc, State: Running, Role: LEADER
I20260929 02:21:38.989275 22674 raft_consensus.cc:2379] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a9858cd94f345d3b54a4cccb3f441b9 in current term 1: Already voted for candidate 37d1d4296f284fe6b440fbe3062779bc in this term.
I20260929 02:21:38.989693 22744 consensus_queue.cc:237] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } }
I20260929 02:21:38.989918 22480 leader_election.cc:302] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a9858cd94f345d3b54a4cccb3f441b9; no voters: 37d1d4296f284fe6b440fbe3062779bc, e9aed72b1efc455e80bc50666a159a81
I20260929 02:21:38.990315 22743 raft_consensus.cc:2726] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260929 02:21:38.994406 22721 tablet.cc:2369] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:21:38.994679 22272 catalog_manager.cc:5495] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1). New cstate: current_term: 1 leader_uuid: "37d1d4296f284fe6b440fbe3062779bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "37d1d4296f284fe6b440fbe3062779bc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 45531 } health_report { overall_health: HEALTHY } } }
I20260929 02:21:39.014423 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:39.008999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.7.170.1:45531)
0929 02:21:39.009234 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:39.009247 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:39.009266 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:21:39.009401 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:39.009405 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:39.009500 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:39.009643 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:21:39.009984 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.010767 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.012413 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.012813 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.012916 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:39.013426 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:21:39.013926 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:21:39.014056 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:39.014128 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 02:21:39.014196 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:21:39.022330 22674 raft_consensus.cc:1261] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Refusing update from remote peer 37d1d4296f284fe6b440fbe3062779bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:39.022346 22544 raft_consensus.cc:1261] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Refusing update from remote peer 37d1d4296f284fe6b440fbe3062779bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:21:39.023146 22750 consensus_queue.cc:1035] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a9858cd94f345d3b54a4cccb3f441b9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 32909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:39.023484 22744 consensus_queue.cc:1035] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9aed72b1efc455e80bc50666a159a81" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:21:39.035218 22753 mvcc.cc:205] Tried to move back new op lower bound from 7334496251986366464 to 7334496251866095616. Current Snapshot: MvccSnapshot[applied={T|T < 7334496251986366464}]
I20260929 02:21:39.035341 22754 mvcc.cc:205] Tried to move back new op lower bound from 7334496251986366464 to 7334496251866095616. Current Snapshot: MvccSnapshot[applied={T|T < 7334496251986366464}]
I20260929 02:21:39.047370 22271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:54672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:39.047617 22271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:54672:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:21:39.051162 22271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Sending DeleteTablet for 3 replicas of tablet a17c1ac4ffba4a48a57c932736e11308
I20260929 02:21:39.052201 22392 tablet_service.cc:1514] Processing DeleteTablet for tablet a17c1ac4ffba4a48a57c932736e11308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:39 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:56486
I20260929 02:21:39.052266 22655 tablet_service.cc:1514] Processing DeleteTablet for tablet a17c1ac4ffba4a48a57c932736e11308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:39 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:48228
I20260929 02:21:39.053066 22762 tablet_replica.cc:330] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: stopping tablet replica
I20260929 02:21:39.052986 22524 tablet_service.cc:1514] Processing DeleteTablet for tablet a17c1ac4ffba4a48a57c932736e11308 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:21:39 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:35506
I20260929 02:21:39.053471 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22309
I20260929 02:21:39.053640 22762 raft_consensus.cc:2229] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:21:39.054471 22762 raft_consensus.cc:2258] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:39.056455 22762 ts_tablet_manager.cc:1905] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:39.065124 22762 ts_tablet_manager.cc:1918] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:39.065374 22762 log.cc:1196] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/a17c1ac4ffba4a48a57c932736e11308
I20260929 02:21:39.065423 22764 tablet_replica.cc:330] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: stopping tablet replica
I20260929 02:21:39.065945 22764 raft_consensus.cc:2229] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:39.066201 22762 ts_tablet_manager.cc:1939] T a17c1ac4ffba4a48a57c932736e11308 P 37d1d4296f284fe6b440fbe3062779bc: Deleting consensus metadata
I20260929 02:21:39.066514 22764 raft_consensus.cc:2258] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:39.068114 22256 catalog_manager.cc:4841] TS 37d1d4296f284fe6b440fbe3062779bc (127.7.170.1:45531): tablet a17c1ac4ffba4a48a57c932736e11308 (table test-table [id=d057f512ad3b49c68f371885ec68a36b]) successfully deleted
I20260929 02:21:39.068470 22764 ts_tablet_manager.cc:1905] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:39.069435 22763 tablet_replica.cc:330] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: stopping tablet replica
I20260929 02:21:39.069849 22763 raft_consensus.cc:2229] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:21:39.070305 22763 raft_consensus.cc:2258] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:21:39.071960 22734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:39.054894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56496 (local address 127.7.170.1:45531)
0929 02:21:39.055061 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:39.055067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:39.055086 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:21:39.055139 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:39.055144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:39.055215 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:39.055326 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:39.057167 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.058240 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.059974 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.060656 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.060765 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:39.064315 (+ 3550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:39.064329 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:39.064347 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:39.064385 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:39.068701 (+ 4316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.069543 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.069552 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.069555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.069713 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.070179 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.070184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.070187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.070705 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:39.071004 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:39.071413 (+ 409us) server_negotiation.cc:299] Negotiation successful
0929 02:21:39.071640 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:21:39.072510 22763 ts_tablet_manager.cc:1905] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:21:39.072610 22452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:39.076045 22764 ts_tablet_manager.cc:1918] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:39.076201 22764 log.cc:1196] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/a17c1ac4ffba4a48a57c932736e11308
I20260929 02:21:39.076793 22764 ts_tablet_manager.cc:1939] T a17c1ac4ffba4a48a57c932736e11308 P 9a9858cd94f345d3b54a4cccb3f441b9: Deleting consensus metadata
I20260929 02:21:39.078737 22257 catalog_manager.cc:4841] TS 9a9858cd94f345d3b54a4cccb3f441b9 (127.7.170.2:32909): tablet a17c1ac4ffba4a48a57c932736e11308 (table test-table [id=d057f512ad3b49c68f371885ec68a36b]) successfully deleted
I20260929 02:21:39.079207 22763 ts_tablet_manager.cc:1918] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:21:39.079347 22763 log.cc:1196] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/a17c1ac4ffba4a48a57c932736e11308
I20260929 02:21:39.079730 22763 ts_tablet_manager.cc:1939] T a17c1ac4ffba4a48a57c932736e11308 P e9aed72b1efc455e80bc50666a159a81: Deleting consensus metadata
I20260929 02:21:39.081149 22259 catalog_manager.cc:4841] TS e9aed72b1efc455e80bc50666a159a81 (127.7.170.3:33693): tablet a17c1ac4ffba4a48a57c932736e11308 (table test-table [id=d057f512ad3b49c68f371885ec68a36b]) successfully deleted
I20260929 02:21:39.167683 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22309
I20260929 02:21:39.182848 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22461
I20260929 02:21:39.195487 22732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:39.183495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35512 (local address 127.7.170.2:32909)
0929 02:21:39.183668 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:39.183673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:39.183727 (+ 54us) server_negotiation.cc:407] Connection header received
0929 02:21:39.183916 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:39.183922 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:39.183999 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:39.184081 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:39.185420 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.186162 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.187422 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.187828 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.187933 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:39.190437 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:39.190449 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:39.190463 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:39.190505 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:39.192871 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.193556 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.193565 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.193568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.193690 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.194021 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.194027 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.194029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.194380 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:39.194642 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:39.194939 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 02:21:39.195183 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:21:39.196100 22584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:39.284478 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22461
I20260929 02:21:39.298046 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22592
I20260929 02:21:39.317243 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:39.298727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.7.170.3:33693)
0929 02:21:39.298926 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:39.298931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:39.298949 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:21:39.299134 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:39.299138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:39.299200 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:39.299280 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:39.300629 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.301478 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.302839 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:39.303208 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:39.303302 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:39.312425 (+ 9123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:39.312441 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:39.312456 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:39.312502 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:39.314994 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.315629 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.315636 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.315639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.315755 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:39.316014 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:39.316017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:39.316019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:39.316355 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:39.316583 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:39.316782 (+ 199us) server_negotiation.cc:299] Negotiation successful
0929 02:21:39.317001 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260929 02:21:39.317827 22715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:39.404891 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22592
I20260929 02:21:39.418640 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22239
I20260929 02:21:39.419659 22302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:39.530308 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22239
I20260929 02:21:39.549178 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--webserver_interface=127.7.170.62
--webserver_port=36381
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:39.865499 22769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:39.865782 22769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:39.865933 22769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:39.872958 22769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:39.873065 22769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:39.873133 22769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:39.873208 22769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:39.873266 22769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:39.883167 22769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=36381
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:39.884549 22769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:39.886067 22769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:39.897477 22774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:39.897571 22777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:39.897496 22775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:39.898370 22769 server_base.cc:1034] running on GCE node
I20260929 02:21:39.899104 22769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:39.900466 22769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:39.901683 22769 hybrid_clock.cc:648] HybridClock initialized: now 1790648499901635 us; error 25 us; skew 500 ppm
I20260929 02:21:39.902050 22769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:39 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648499, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:39.905033 22769 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:39.906397 22769 webserver.cc:460] Webserver started at http://127.7.170.62:36381/ using document root <none> and password file <none>
I20260929 02:21:39.907040 22769 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:39.907136 22769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:39.912590 22769 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:39.915427 22784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:39.916756 22769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:39.916915 22769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e742251329a84c839c44af920ccf4baf"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "d2c1ead3bc5a37eede10aa11bdede8cb"
server_key_iv: "0dbc23d4159f394968f6f260ecc744fc"
server_key_version: "encryptionkey@0"
I20260929 02:21:39.917892 22769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:39.930406 22769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:39.934526 22769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:39.934782 22769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:39.951431 22769 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38141
I20260929 02:21:39.951458 22835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38141 every 8 connection(s)
I20260929 02:21:39.952867 22769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:39.959652 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22769
I20260929 02:21:39.959700 22836 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:21:39.960424 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:45531
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=37067
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:39.962977 22836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap starting.
I20260929 02:21:39.988021 22836 log.cc:824] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:40.008040 22836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:21:40.008666 22836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf: Bootstrap complete.
I20260929 02:21:40.021849 22836 raft_consensus.cc:348] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:40.023096 22836 raft_consensus.cc:729] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e742251329a84c839c44af920ccf4baf, State: Initialized, Role: FOLLOWER
I20260929 02:21:40.023793 22836 consensus_queue.cc:260] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:40.024063 22836 raft_consensus.cc:388] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:40.024179 22836 raft_consensus.cc:482] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:40.024350 22836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:21:40.030207 22836 raft_consensus.cc:504] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:40.030784 22836 leader_election.cc:302] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e742251329a84c839c44af920ccf4baf; no voters:
I20260929 02:21:40.031469 22836 leader_election.cc:288] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 02:21:40.031599 22840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 3 FOLLOWER]: Leader election won for term 3
I20260929 02:21:40.032626 22840 raft_consensus.cc:686] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [term 3 LEADER]: Becoming Leader. State: Replica: e742251329a84c839c44af920ccf4baf, State: Running, Role: LEADER
I20260929 02:21:40.033147 22840 consensus_queue.cc:237] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } }
I20260929 02:21:40.033520 22836 sys_catalog.cc:564] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:40.035360 22841 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:40.035698 22841 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:40.036239 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e742251329a84c839c44af920ccf4baf. Latest consensus state: current_term: 3 leader_uuid: "e742251329a84c839c44af920ccf4baf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e742251329a84c839c44af920ccf4baf" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38141 } } }
I20260929 02:21:40.036579 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P e742251329a84c839c44af920ccf4baf [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:40.039162 22845 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:40.047621 22845 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:40.049082 22845 catalog_manager.cc:1259] Loaded cluster ID: 9695d43113694c0897d04e66d5ca8b25
I20260929 02:21:40.049225 22845 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260929 02:21:40.051177 22845 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc8af6189ad google::LogMessage::Fail() at ??:0
@ 0x7fc8af61b88a google::LogMessage::SendToLog() at ??:0
@ 0x7fc8af6184de google::LogMessage::Flush() at ??:0
@ 0x7fc8af619a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc8c504b512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc8c5048976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc8c50d0610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc8c50d0439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
I20260929 02:21:40.079981 22858 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fc8b2264345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc8b226791c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc8b2267799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc8b2242eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc8b551c6db start_thread at ??:0
@ 0x7fc8ae4b261f clone at ??:0
W20260929 02:21:40.361167 22838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:40.361454 22838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:40.361582 22838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:40.368918 22838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:40.369043 22838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:40.369186 22838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:40.378824 22838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:45531
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=37067
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:40.380081 22838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:40.381384 22838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:40.393869 22877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:40.393906 22878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:40.393929 22880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:40.394873 22838 server_base.cc:1034] running on GCE node
I20260929 02:21:40.395447 22838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:40.396368 22838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:40.397583 22838 hybrid_clock.cc:648] HybridClock initialized: now 1790648500397525 us; error 39 us; skew 500 ppm
I20260929 02:21:40.397889 22838 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:40 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648500, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:40.400746 22838 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:40.402099 22838 webserver.cc:460] Webserver started at http://127.7.170.1:37067/ using document root <none> and password file <none>
I20260929 02:21:40.402720 22838 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:40.402813 22838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:40.408195 22838 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260929 02:21:40.410879 22887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.412365 22838 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:40.412518 22838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "37d1d4296f284fe6b440fbe3062779bc"
format_stamp: "Formatted at 2026-09-29 02:21:34 on dist-test-slave-8cs2"
server_key: "a7ddb116109ce89e1652964e10c6ac56"
server_key_iv: "60686093683b0e84b43f8e54180732cc"
server_key_version: "encryptionkey@0"
I20260929 02:21:40.413582 22838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:40.432839 22838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:40.436667 22838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:40.436919 22838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:40.437970 22838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:40.439378 22838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:40.439466 22838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.439576 22838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:40.439648 22838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.470150 22838 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:45531
I20260929 02:21:40.470227 22999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:45531 every 8 connection(s)
I20260929 02:21:40.471540 22838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:40.476265 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.474376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:40073)
0929 02:21:40.475117 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.475332 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":275,"threads_started":1}
W20260929 02:21:40.476981 23000 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:38141 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:38141: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
I20260929 02:21:40.477758 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.477455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:50297)
0929 02:21:40.477587 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.477657 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260929 02:21:40.478500 23002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.478214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:37047)
0929 02:21:40.478327 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.478399 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:21:40.478710 23000 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:21:40.479285 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 22838
I20260929 02:21:40.479825 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:32909
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=38215
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:40.797387 23003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:40.797665 23003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:40.797793 23003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:40.804790 23003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:40.804888 23003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:40.805060 23003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:40.814576 23003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:32909
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=38215
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:40.815774 23003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:40.817075 23003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:40.830566 23009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:40.830708 23008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:40.830682 23011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:40.831444 23003 server_base.cc:1034] running on GCE node
I20260929 02:21:40.832037 23003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:40.832906 23003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:40.834142 23003 hybrid_clock.cc:648] HybridClock initialized: now 1790648500834081 us; error 55 us; skew 500 ppm
I20260929 02:21:40.834442 23003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:40 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648500, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:40.837380 23003 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:40.838742 23003 webserver.cc:460] Webserver started at http://127.7.170.2:38215/ using document root <none> and password file <none>
I20260929 02:21:40.839413 23003 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:40.839522 23003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:40.844733 23003 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:21:40.847538 23018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.848866 23003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:21:40.849067 23003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "9a9858cd94f345d3b54a4cccb3f441b9"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "74306fd30fa03ca3871eec5219c4ae10"
server_key_iv: "d12d002715f5d70e8ec38cc3f5307629"
server_key_version: "encryptionkey@0"
I20260929 02:21:40.850047 23003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:40.871183 23003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:40.875316 23003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:40.875576 23003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:40.876499 23003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:40.877796 23003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:40.877897 23003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.878007 23003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:40.878074 23003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:40.907366 23003 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:32909
I20260929 02:21:40.907423 23130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:32909 every 8 connection(s)
I20260929 02:21:40.908725 23003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:40.910996 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23003
I20260929 02:21:40.911764 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:33693
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:38141
--webserver_port=46837
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:21:40.913997 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.911982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:46641)
0929 02:21:40.912790 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.913071 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":297,"threads_started":1}
W20260929 02:21:40.914753 23131 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:38141 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:38141: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
I20260929 02:21:40.915635 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.915294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:36865)
0929 02:21:40.915462 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.915530 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:21:40.916477 23133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:40.916170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:46575)
0929 02:21:40.916281 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:40.916350 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:21:40.916715 23131 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:21:41.226529 23134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:41.226812 23134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:41.226945 23134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:41.234076 23134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:41.234175 23134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:41.234313 23134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:41.243763 23134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:33693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=46837
--tserver_master_addrs=127.7.170.62:38141
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:41.245054 23134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:41.246361 23134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:41.259073 23139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:41.259101 23140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:41.259614 23134 server_base.cc:1034] running on GCE node
W20260929 02:21:41.259648 23142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:41.260370 23134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:41.261363 23134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:41.262562 23134 hybrid_clock.cc:648] HybridClock initialized: now 1790648501262514 us; error 40 us; skew 500 ppm
I20260929 02:21:41.262876 23134 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:41 dist-test-slave-8cs2 krb5kdc[21687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648501, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:41.266090 23134 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:41.267513 23134 webserver.cc:460] Webserver started at http://127.7.170.3:46837/ using document root <none> and password file <none>
I20260929 02:21:41.268180 23134 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:41.268276 23134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:41.273413 23134 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:21:41.275967 23149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:41.277264 23134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:41.277407 23134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "e9aed72b1efc455e80bc50666a159a81"
format_stamp: "Formatted at 2026-09-29 02:21:35 on dist-test-slave-8cs2"
server_key: "4dff5e9a17a1c53a62bbdc66704a134c"
server_key_iv: "2edb6ffe41e873b8e3e3e405b0a744a1"
server_key_version: "encryptionkey@0"
I20260929 02:21:41.278322 23134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:41.297578 23134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:41.301321 23134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:41.301558 23134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:41.302479 23134 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:41.303746 23134 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:41.303830 23134 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:41.303933 23134 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:41.303997 23134 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:41.332790 23134 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33693
I20260929 02:21:41.332855 23261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33693 every 8 connection(s)
I20260929 02:21:41.334194 23134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:21:41.338696 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:41.336976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:46849)
0929 02:21:41.337662 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:41.337856 (+ 194us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":247,"threads_started":1}
W20260929 02:21:41.339298 23262 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:38141 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:38141: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
I20260929 02:21:41.339998 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:41.339710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:59787)
0929 02:21:41.339829 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:41.339889 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:21:41.340742 23264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:41.340451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:50613)
0929 02:21:41.340576 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:41.340645 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 02:21:41.340986 23262 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:21:41.344369 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23134
I20260929 02:21:41.480001 23266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:41.479214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:44489)
0929 02:21:41.479670 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:41.479767 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":234,"threads_started":1}
I20260929 02:21:41.918076 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:41.917298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:36117)
0929 02:21:41.917746 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:41.917837 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":216,"threads_started":1}
I20260929 02:21:42.342135 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:42.341357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:39827)
0929 02:21:42.341801 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:42.341899 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260929 02:21:42.481698 23269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:42.480999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:54945)
0929 02:21:42.481419 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:42.481503 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":200,"threads_started":1}
I20260929 02:21:42.919669 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:42.918926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:58351)
0929 02:21:42.919397 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:42.919479 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":236,"threads_started":1}
I20260929 02:21:43.343675 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:43.342951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:33655)
0929 02:21:43.343413 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:43.343497 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":226,"threads_started":1}
I20260929 02:21:43.483209 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:43.482497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:44771)
0929 02:21:43.482954 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:43.483036 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":215,"threads_started":1}
I20260929 02:21:43.921353 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:43.920543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:45055)
0929 02:21:43.921047 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:43.921136 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":243,"threads_started":1}
I20260929 02:21:44.345250 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:44.344529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:33023)
0929 02:21:44.345001 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:44.345078 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":231,"threads_started":1}
I20260929 02:21:44.484817 23275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:44.484105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:35397)
0929 02:21:44.484539 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:44.484626 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":210,"threads_started":1}
I20260929 02:21:44.923051 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:44.922303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:48121)
0929 02:21:44.922761 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:44.922837 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":249,"threads_started":1}
I20260929 02:21:45.346788 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:45.346114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:44553)
0929 02:21:45.346525 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:45.346604 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":197,"threads_started":1}
I20260929 02:21:45.486384 23278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:45.485693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:56887)
0929 02:21:45.486136 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:45.486214 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":234,"threads_started":1}
I20260929 02:21:45.924790 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:45.924039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:37181)
0929 02:21:45.924529 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:45.924615 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":241,"threads_started":1}
I20260929 02:21:46.348604 23280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:46.347800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:35065)
0929 02:21:46.348322 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:46.348427 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":278,"threads_started":1}
I20260929 02:21:46.488005 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:46.487277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:41141)
0929 02:21:46.487739 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:46.487835 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":242,"threads_started":1}
I20260929 02:21:46.926317 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:46.925639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:44061)
0929 02:21:46.926068 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:46.926155 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":222,"threads_started":1}
I20260929 02:21:47.350238 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:47.349495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:38135)
0929 02:21:47.349984 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:47.350078 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":250,"threads_started":1}
I20260929 02:21:47.489669 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:47.488887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:58181)
0929 02:21:47.489376 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:47.489457 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":192,"threads_started":1}
I20260929 02:21:47.928068 23285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:47.927237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:33607)
0929 02:21:47.927744 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:47.927835 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":232,"threads_started":1}
I20260929 02:21:48.351994 23286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:48.351164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:56961)
0929 02:21:48.351689 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:48.351774 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":243,"threads_started":1}
I20260929 02:21:48.491362 23287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:48.490580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:33789)
0929 02:21:48.491108 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:48.491196 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":258,"threads_started":1}
I20260929 02:21:48.929693 23288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:48.928990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:44915)
0929 02:21:48.929445 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:48.929530 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":217,"threads_started":1}
I20260929 02:21:49.353603 23289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:49.352877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:38629)
0929 02:21:49.353354 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:49.353437 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":247,"threads_started":1}
I20260929 02:21:49.493101 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:49.492297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:60677)
0929 02:21:49.492777 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:49.492880 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":238,"threads_started":1}
I20260929 02:21:49.931339 23291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:49.930569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:52179)
0929 02:21:49.931057 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:49.931147 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":234,"threads_started":1}
I20260929 02:21:50.355160 23292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:50.354426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:56339)
0929 02:21:50.354903 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:50.354991 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":233,"threads_started":1}
I20260929 02:21:50.494742 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:50.494015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:37977)
0929 02:21:50.494488 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:50.494576 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":243,"threads_started":1}
I20260929 02:21:50.933125 23294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:50.932305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:51381)
0929 02:21:50.932785 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:50.932886 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":248,"threads_started":1}
I20260929 02:21:51.356691 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:51.356025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:56849)
0929 02:21:51.356454 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:51.356537 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260929 02:21:51.496261 23296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:51.495549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:57159)
0929 02:21:51.495993 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:51.496084 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":214,"threads_started":1}
I20260929 02:21:51.935024 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:51.934199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:44601)
0929 02:21:51.934699 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:51.934800 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":263,"threads_started":1}
I20260929 02:21:52.358304 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:52.357585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:34933)
0929 02:21:52.358031 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:52.358121 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":200,"threads_started":1}
I20260929 02:21:52.497989 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:52.497225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:54681)
0929 02:21:52.497665 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:52.497776 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":204,"threads_started":1}
I20260929 02:21:52.936750 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:52.936007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:34323)
0929 02:21:52.936498 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:52.936591 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":230,"threads_started":1}
I20260929 02:21:53.359932 23301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:53.359256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:48887)
0929 02:21:53.359691 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:53.359780 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":216,"threads_started":1}
I20260929 02:21:53.499836 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:53.498981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:49379)
0929 02:21:53.499528 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:53.499634 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":254,"threads_started":1}
I20260929 02:21:53.938591 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:53.937748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:41837)
0929 02:21:53.938266 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:53.938373 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":239,"threads_started":1}
I20260929 02:21:54.361660 23304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:54.360828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:59185)
0929 02:21:54.361359 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:54.361473 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":279,"threads_started":1}
I20260929 02:21:54.501557 23305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:54.500768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:33745)
0929 02:21:54.501277 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:54.501374 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":287,"threads_started":1}
I20260929 02:21:54.940284 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:54.939542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:41731)
0929 02:21:54.940018 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:54.940106 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":230,"threads_started":1}
I20260929 02:21:55.363291 23307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:55.362542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:44081)
0929 02:21:55.363024 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:55.363111 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":258,"threads_started":1}
I20260929 02:21:55.503513 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:55.502733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.1:52661)
0929 02:21:55.503210 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:55.503310 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":233,"threads_started":1}
I20260929 02:21:55.942111 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:55.941376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.2:53711)
0929 02:21:55.941804 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:55.941907 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":224,"threads_started":1}
I20260929 02:21:56.353279 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 22838
I20260929 02:21:56.365365 23311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:56.364427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38141 (local address 127.7.170.3:59287)
0929 02:21:56.364997 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:56.365112 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:38141: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":234,"threads_started":1}
I20260929 02:21:56.372331 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:56.354526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54004 (local address 127.7.170.1:45531)
0929 02:21:56.355159 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:56.355180 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:56.355229 (+ 49us) server_negotiation.cc:407] Connection header received
0929 02:21:56.357266 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:56.357277 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:56.357731 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:56.358059 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:56.359491 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.360868 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.362233 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.362973 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.363116 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:56.366235 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:56.366244 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:56.366261 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:56.366304 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:56.369206 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.369917 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.369924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.369931 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.370137 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.370482 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.370484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.370486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.371064 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:56.371299 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:56.371673 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 02:21:56.372048 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":290,"threads_started":1}
I20260929 02:21:56.373764 22995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:56.459100 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 22838
I20260929 02:21:56.473964 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23003
I20260929 02:21:56.493247 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:56.475014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52352 (local address 127.7.170.2:32909)
0929 02:21:56.475579 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:56.475597 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:56.475644 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:21:56.477309 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:56.477320 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:56.477779 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:56.478126 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:56.479625 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.480916 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.482276 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.483042 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.483246 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:56.486124 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:56.486134 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:56.486152 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:56.486197 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:56.489362 (+ 3165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.490224 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.490233 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.490241 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.490451 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.490868 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.490872 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.490874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.491466 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:56.491769 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:56.492265 (+ 496us) server_negotiation.cc:299] Negotiation successful
0929 02:21:56.492729 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":250,"threads_started":1}
I20260929 02:21:56.494709 23126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:56.581065 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23003
I20260929 02:21:56.596587 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23134
I20260929 02:21:56.614835 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:56.597573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60898 (local address 127.7.170.3:33693)
0929 02:21:56.598143 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:56.598157 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:56.598201 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:21:56.599862 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:56.599873 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:56.600401 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:56.600847 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:56.602522 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.603769 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.605202 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:56.605943 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:56.606098 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:56.608563 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:56.608572 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:56.608589 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:56.608637 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:56.611559 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.612305 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.612316 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.612326 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.612490 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:56.612820 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:56.612823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:56.612825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:56.613444 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:56.613721 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:56.614160 (+ 439us) server_negotiation.cc:299] Negotiation successful
0929 02:21:56.614532 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":271,"threads_started":1}
I20260929 02:21:56.616195 23257 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:21:56.699581 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23134
2026-09-29T02:21:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24069 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:21:56 dist-test-slave-8cs2 krb5kdc[23319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:21:56 dist-test-slave-8cs2 krb5kdc[23319](info): set up 2 sockets
Sep 29 02:21:56 dist-test-slave-8cs2 krb5kdc[23319](info): commencing operation
krb5kdc: starting...
W20260929 02:21:57.717617 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.981s user 0.007s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:21:57 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:21:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:21:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:21:57.969427 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:43475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:58.300549 23332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:58.300868 23332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:58.301024 23332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:58.308414 23332 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:21:58.308521 23332 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:21:58.308562 23332 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:21:58.308605 23332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:21:58.308652 23332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:58.308750 23332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:58.308806 23332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:21:58.308862 23332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:21:58.319556 23332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:43475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:58.320917 23332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:58.322322 23332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:58.334076 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:58.334122 23340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:58.334041 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:58.338552 23332 server_base.cc:1034] running on GCE node
I20260929 02:21:58.339418 23332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:58.340984 23332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:58.342226 23332 hybrid_clock.cc:648] HybridClock initialized: now 1790648518342166 us; error 31 us; skew 500 ppm
I20260929 02:21:58.342582 23332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:58 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648518, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:58.345731 23332 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:21:58.347334 23332 webserver.cc:460] Webserver started at http://127.7.170.62:44171/ using document root <none> and password file <none>
I20260929 02:21:58.348136 23332 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:58.348240 23332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:58.348558 23332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:58.351585 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "3fe8752ad98e43ceaeffae5099214cc5"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "7bf602c87a214fd0f40ad72356fe09f6"
server_key_iv: "7b283df76dfe85f74f5c53ab979926dd"
server_key_version: "encryptionkey@0"
I20260929 02:21:58.352450 23332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "3fe8752ad98e43ceaeffae5099214cc5"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "7bf602c87a214fd0f40ad72356fe09f6"
server_key_iv: "7b283df76dfe85f74f5c53ab979926dd"
server_key_version: "encryptionkey@0"
I20260929 02:21:58.357658 23332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 02:21:58.361233 23347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:58.362671 23332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:21:58.362830 23332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "3fe8752ad98e43ceaeffae5099214cc5"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "7bf602c87a214fd0f40ad72356fe09f6"
server_key_iv: "7b283df76dfe85f74f5c53ab979926dd"
server_key_version: "encryptionkey@0"
I20260929 02:21:58.363071 23332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:58.393536 23332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:58.398137 23332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:58.398422 23332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:58.416705 23332 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:43475
I20260929 02:21:58.416724 23398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:43475 every 8 connection(s)
I20260929 02:21:58.418164 23332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:21:58.422099 23399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:21:58.423689 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23332
I20260929 02:21:58.423938 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:21:58.424734 7848 external_mini_cluster.cc:1428] Setting key 51dc28e2500b65fade20fd097cd423dc
I20260929 02:21:58.435031 23399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap starting.
I20260929 02:21:58.439056 23399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Neither blocks nor log segments found. Creating new log.
Sep 29 02:21:58 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:58.440253 23399 log.cc:824] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:21:58.443442 23399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: No bootstrap required, opened a new log
I20260929 02:21:58.449119 23402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:58.427005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.7.170.62:43475)
0929 02:21:58.427889 (+ 884us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:58.427902 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:58.427958 (+ 56us) server_negotiation.cc:407] Connection header received
0929 02:21:58.429680 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:58.429704 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:58.430134 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:58.430453 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:58.432289 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:58.433754 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:58.435614 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:58.436350 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:58.436485 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:58.441481 (+ 4996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:58.441494 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:58.441521 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:58.441576 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:58.444542 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:58.445458 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:58.445464 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:58.445471 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:58.445640 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:58.446104 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:58.446109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:58.446111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:58.446753 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:58.447027 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:58.447502 (+ 475us) server_negotiation.cc:299] Negotiation successful
0929 02:21:58.447964 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":413,"threads_started":1}
I20260929 02:21:58.452586 23399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:21:58.452975 23399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:21:58.453070 23399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Initialized, Role: FOLLOWER
I20260929 02:21:58.453616 23399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:21:58.453840 23399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:21:58.453963 23399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:21:58.454118 23399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:21:58.456681 23399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:21:58.457363 23399 leader_election.cc:302] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fe8752ad98e43ceaeffae5099214cc5; no voters:
I20260929 02:21:58.457918 23399 leader_election.cc:288] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:21:58.458048 23404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:21:58.458657 23404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 LEADER]: Becoming Leader. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Running, Role: LEADER
I20260929 02:21:58.459219 23404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:21:58.459895 23399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:21:58.461881 23406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fe8752ad98e43ceaeffae5099214cc5. Latest consensus state: current_term: 1 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:21:58.462286 23406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:58.462875 23405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:21:58.463184 23410 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:21:58.463217 23405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:21:58.468523 23410 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:21:58.478487 23410 catalog_manager.cc:1347] Generated new cluster ID: b5e46e07437f47a5a32aacc3930f22a9
I20260929 02:21:58.478618 23410 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:21:58.504985 23410 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:21:58.506500 23410 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:21:58.518519 23410 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Generated new TSK 0
I20260929 02:21:58.519402 23410 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:21:58.597107 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:58.949980 23425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:58.950268 23425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:58.950397 23425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:58.957886 23425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:58.958012 23425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:58.958161 23425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:21:58.969069 23425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:58.970422 23425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:58.971736 23425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:58.984604 23431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:58.984647 23430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:58.984567 23433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:58.985333 23425 server_base.cc:1034] running on GCE node
I20260929 02:21:58.985918 23425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:58.986830 23425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:58.988014 23425 hybrid_clock.cc:648] HybridClock initialized: now 1790648518987967 us; error 35 us; skew 500 ppm
I20260929 02:21:58.988299 23425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:58 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648518, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:58.991168 23425 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:21:58.992578 23425 webserver.cc:460] Webserver started at http://127.7.170.1:46311/ using document root <none> and password file <none>
I20260929 02:21:58.993281 23425 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:58.993397 23425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:58.993695 23425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:58.996574 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "f25762067c8f447c9f0a589bc17b6886"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "de7ca7749ad8cf4dc007eb3e0927e8c9"
server_key_iv: "f66687778646e273b065046e6aba9b36"
server_key_version: "encryptionkey@0"
I20260929 02:21:58.997326 23425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "f25762067c8f447c9f0a589bc17b6886"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "de7ca7749ad8cf4dc007eb3e0927e8c9"
server_key_iv: "f66687778646e273b065046e6aba9b36"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.002400 23425 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260929 02:21:59.005697 23440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:59.007031 23425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:21:59.007179 23425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "f25762067c8f447c9f0a589bc17b6886"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "de7ca7749ad8cf4dc007eb3e0927e8c9"
server_key_iv: "f66687778646e273b065046e6aba9b36"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.007349 23425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:59.025980 23425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:59.030068 23425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:59.030383 23425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:59.031350 23425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:59.032693 23425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:59.032766 23425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:59.032900 23425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:59.032974 23425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260929 02:21:59.063572 23425 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46555
I20260929 02:21:59.063616 23552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46555 every 8 connection(s)
I20260929 02:21:59.064891 23425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:21:59.065933 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23425
I20260929 02:21:59.066182 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:21:59.067008 7848 external_mini_cluster.cc:1428] Setting key f4568d5eb0f2e567ea2dc114230dc2e3
Sep 29 02:21:59 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648518, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:59.087863 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:59.068288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:59651 (local address 127.7.170.62:43475)
0929 02:21:59.068709 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:59.068714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:59.070021 (+ 1307us) server_negotiation.cc:407] Connection header received
0929 02:21:59.072470 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:59.072475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:59.072544 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:59.072639 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:59.074681 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.075404 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.077556 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.077950 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.078061 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:59.081741 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:59.081749 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:59.081753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:59.081799 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:59.084789 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:59.085549 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:59.085554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:59.085558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:59.085701 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:59.086124 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:59.086129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:59.086132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:59.086457 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:59.086591 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:59.087301 (+ 710us) server_negotiation.cc:299] Negotiation successful
0929 02:21:59.087516 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":222,"threads_started":1}
I20260929 02:21:59.088776 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:59.068835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:59651)
0929 02:21:59.069780 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:59.069845 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:59.072181 (+ 2336us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:59.072966 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:59.072979 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:59.073699 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:59.074508 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:59.074533 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.075574 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.075580 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:59.077430 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:59.077444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.078181 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.078187 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:59.078457 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:59.079186 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:59.079217 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:59.081481 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:59.084966 (+ 3485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:59.084974 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:59.084989 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:59.085412 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:59.085789 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:59.085792 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:59.085804 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:59.086006 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:59.086561 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:59.086567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:59.086899 (+ 332us) client_negotiation.cc:769] Sending connection context
0929 02:21:59.087161 (+ 262us) client_negotiation.cc:240] Negotiation successful
0929 02:21:59.087556 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":364,"threads_started":1}
I20260929 02:21:59.090375 23553 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:21:59.090693 23553 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:59.091398 23553 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:21:59.093855 23364 ts_manager.cc:194] Registered new tserver with Master: f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:21:59.095497 23364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:59651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:21:59.124629 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:59.450591 23559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:59.450860 23559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:59.450985 23559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:59.458060 23559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:59.458161 23559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:59.458307 23559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:21:59.468062 23559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:59.469408 23559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:59.470692 23559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:59.482892 23564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:59.482944 23567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:59.482945 23565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:59.484025 23559 server_base.cc:1034] running on GCE node
I20260929 02:21:59.484601 23559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:59.485522 23559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:59.486708 23559 hybrid_clock.cc:648] HybridClock initialized: now 1790648519486661 us; error 45 us; skew 500 ppm
I20260929 02:21:59.486996 23559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:59 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648519, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:59.489950 23559 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:21:59.491320 23559 webserver.cc:460] Webserver started at http://127.7.170.2:39015/ using document root <none> and password file <none>
I20260929 02:21:59.491990 23559 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:59.492106 23559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:59.492391 23559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:59.495280 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "98f15783d7e9402da21d8c1367cb6c3b"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "e2e511216eec6a59d415ff9b66d9e139"
server_key_iv: "e7ec82fc44811f4047217bc31b0f6562"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.496025 23559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "98f15783d7e9402da21d8c1367cb6c3b"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "e2e511216eec6a59d415ff9b66d9e139"
server_key_iv: "e7ec82fc44811f4047217bc31b0f6562"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.501258 23559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:21:59.504505 23574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:59.505854 23559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:21:59.505997 23559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "98f15783d7e9402da21d8c1367cb6c3b"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "e2e511216eec6a59d415ff9b66d9e139"
server_key_iv: "e7ec82fc44811f4047217bc31b0f6562"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.506160 23559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:21:59.529181 23559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:21:59.533492 23559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:21:59.533764 23559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:21:59.534734 23559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:21:59.536043 23559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:21:59.536123 23559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:59.536240 23559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:21:59.536291 23559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:21:59.565178 23559 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34507
I20260929 02:21:59.565218 23686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34507 every 8 connection(s)
I20260929 02:21:59.566526 23559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:21:59.567895 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23559
I20260929 02:21:59.568060 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:21:59.568473 7848 external_mini_cluster.cc:1428] Setting key c8cf3b0b44c64073fe3fd5b14cf3cb13
Sep 29 02:21:59 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648519, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:21:59.586525 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:59.569092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54407 (local address 127.7.170.62:43475)
0929 02:21:59.569288 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:21:59.569294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:21:59.570882 (+ 1588us) server_negotiation.cc:407] Connection header received
0929 02:21:59.572711 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:21:59.572716 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:21:59.572800 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:21:59.572888 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:21:59.574394 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.575092 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.576591 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.577079 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.577190 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:59.580902 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:21:59.580907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:21:59.580910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:21:59.580980 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:21:59.583472 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:59.584304 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:59.584307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:59.584310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:59.584425 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:21:59.584848 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:21:59.584851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:21:59.584853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:21:59.585158 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:21:59.585317 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0929 02:21:59.586101 (+ 784us) server_negotiation.cc:299] Negotiation successful
0929 02:21:59.586275 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:21:59.587926 23689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:21:59.569756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:54407)
0929 02:21:59.570737 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0929 02:21:59.570776 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:21:59.572456 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:21:59.573153 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:21:59.573162 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:21:59.573681 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:21:59.574237 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:59.574255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.575237 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.575240 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:59.576459 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:21:59.576471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:21:59.577292 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:21:59.577300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:21:59.577524 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:21:59.578188 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:21:59.578212 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:21:59.580657 (+ 2445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:21:59.583610 (+ 2953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:59.583620 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:59.583639 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:59.584208 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:59.584517 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:21:59.584521 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:21:59.584524 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:21:59.584758 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:21:59.585299 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:21:59.585306 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:21:59.585709 (+ 403us) client_negotiation.cc:769] Sending connection context
0929 02:21:59.586044 (+ 335us) client_negotiation.cc:240] Negotiation successful
0929 02:21:59.586571 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":449,"threads_started":1}
I20260929 02:21:59.589299 23687 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:21:59.589668 23687 heartbeater.cc:464] Registering TS with master...
I20260929 02:21:59.590559 23687 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:21:59.592118 23364 ts_manager.cc:194] Registered new tserver with Master: 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507)
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
I20260929 02:21:59.592970 23364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:54407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:21:59.623973 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:21:59.945601 23692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:21:59.945910 23692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:21:59.946015 23692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:21:59.953542 23692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:21:59.953653 23692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:21:59.953768 23692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:21:59.963598 23692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:21:59.964855 23692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:21:59.966226 23692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:21:59.979019 23700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:59.978981 23698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:21:59.979131 23697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:21:59.980070 23692 server_base.cc:1034] running on GCE node
I20260929 02:21:59.980693 23692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:21:59.981690 23692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:21:59.982877 23692 hybrid_clock.cc:648] HybridClock initialized: now 1790648519982815 us; error 52 us; skew 500 ppm
I20260929 02:21:59.983187 23692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:21:59 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648519, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:21:59.986217 23692 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:21:59.987641 23692 webserver.cc:460] Webserver started at http://127.7.170.3:41579/ using document root <none> and password file <none>
I20260929 02:21:59.988310 23692 fs_manager.cc:359] Metadata directory not provided
I20260929 02:21:59.988433 23692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:21:59.988718 23692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:21:59.991652 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "b8bb5ec174ca504aa57c4588216973bf"
server_key_iv: "12c483cf3884e0ff071c0bfa333629ce"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.992380 23692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "b8bb5ec174ca504aa57c4588216973bf"
server_key_iv: "12c483cf3884e0ff071c0bfa333629ce"
server_key_version: "encryptionkey@0"
I20260929 02:21:59.997481 23692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 02:22:00.000859 23707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:00.002178 23692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:22:00.002329 23692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "b8bb5ec174ca504aa57c4588216973bf"
server_key_iv: "12c483cf3884e0ff071c0bfa333629ce"
server_key_version: "encryptionkey@0"
I20260929 02:22:00.002513 23692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:00.028877 23692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:00.032756 23692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:00.033042 23692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:00.033989 23692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:00.035281 23692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:00.035357 23692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:00.035480 23692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:00.035532 23692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:00.065771 23692 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33877
I20260929 02:22:00.065774 23819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33877 every 8 connection(s)
I20260929 02:22:00.067344 23692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:22:00.077292 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23692
I20260929 02:22:00.077515 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:22:00.078111 7848 external_mini_cluster.cc:1428] Setting key 929174eb5ee07a608f566fa20b435995
Sep 29 02:22:00 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648519, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:22:00.086865 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.070129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:50615 (local address 127.7.170.62:43475)
0929 02:22:00.070305 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.070310 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.071480 (+ 1170us) server_negotiation.cc:407] Connection header received
0929 02:22:00.073310 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.073315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.073385 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.073461 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:00.074884 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.075613 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.077178 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.077777 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.077903 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.081801 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:00.081806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:00.081809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:00.081843 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:00.084271 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.084969 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.084974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.084978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.085104 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.085475 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.085478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.085480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.085720 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:00.085838 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.086449 (+ 611us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.086610 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:22:00.087896 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.070603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:50615)
0929 02:22:00.071307 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.071343 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.073051 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.073648 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.073656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.074180 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:00.074749 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.074767 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.075721 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.075725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.077038 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.077050 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.077990 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.077995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.078198 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.078911 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:00.078942 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:00.081605 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:00.084401 (+ 2796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:00.084408 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:00.084422 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:00.084847 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:00.085187 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:00.085190 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:00.085193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:00.085393 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:00.085822 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:00.085827 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:00.086155 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.086374 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.086767 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":274,"threads_started":1}
I20260929 02:22:00.089277 23820 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:22:00.089597 23820 heartbeater.cc:464] Registering TS with master...
I20260929 02:22:00.090296 23820 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:22:00.091719 23364 ts_manager.cc:194] Registered new tserver with Master: 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
I20260929 02:22:00.092610 23364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:50615
I20260929 02:22:00.093611 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:22:00.099143 23553 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:00.110517 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.097890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50854 (local address 127.7.170.62:43475)
0929 02:22:00.098052 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.098058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.098434 (+ 376us) server_negotiation.cc:407] Connection header received
0929 02:22:00.098597 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.098600 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.098660 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.098739 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:00.100427 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.101316 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.102754 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.103185 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.103294 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.105681 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:00.105686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:00.105690 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:00.105721 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:00.108160 (+ 2439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.108786 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.108790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.108792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.108912 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.109268 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.109271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.109273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.109601 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:00.109780 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.110075 (+ 295us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.110245 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:22:00.114589 23363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50854:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:22:00.121754 23363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:22:00.145170 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.134954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53096 (local address 127.7.170.2:34507)
0929 02:22:00.136117 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.136127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.137357 (+ 1230us) server_negotiation.cc:407] Connection header received
0929 02:22:00.137653 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.137661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.137894 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.138060 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:00.139068 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.140340 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.142565 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.143938 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.144060 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.144268 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.144614 (+ 346us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.144798 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1002,"thread_start_us":874,"threads_started":1}
I20260929 02:22:00.145284 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.134319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.7.170.3:33877)
0929 02:22:00.135658 (+ 1339us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.135669 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.136807 (+ 1138us) server_negotiation.cc:407] Connection header received
0929 02:22:00.137129 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.137135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.137352 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.137539 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:00.138449 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.139978 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.142887 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.144250 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.144382 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.144626 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.144788 (+ 162us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.144980 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1129,"thread_start_us":966,"threads_started":1}
I20260929 02:22:00.145709 23834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.134110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33877 (local address 127.0.0.1:49994)
0929 02:22:00.136645 (+ 2535us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.136687 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.136866 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.137814 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.137820 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.137854 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:00.138262 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.138276 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.140103 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.140109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.142717 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.142732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.144454 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.144460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.144621 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.144653 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.144735 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.144971 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2308,"mutex_wait_us":225,"thread_start_us":2914,"threads_started":1}
I20260929 02:22:00.145987 23832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.134440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34507 (local address 127.0.0.1:53096)
0929 02:22:00.137260 (+ 2820us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.137277 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.137460 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.138356 (+ 896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.138360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.138382 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:00.138820 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.138835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.140458 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.140462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.142427 (+ 1965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.142439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.144132 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.144139 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.144323 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.144376 (+ 53us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.145397 (+ 1021us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.145543 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2710,"mutex_wait_us":19,"thread_start_us":1848,"threads_started":1}
I20260929 02:22:00.148355 23836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.137367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:50560)
0929 02:22:00.139161 (+ 1794us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.139178 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.139290 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.139807 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.139812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.139838 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:00.140218 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.140228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.142132 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.142137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.143699 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.143710 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.147695 (+ 3985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.147700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.147817 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.147841 (+ 24us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.147923 (+ 82us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.148049 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1671,"thread_start_us":1542,"threads_started":1}
I20260929 02:22:00.148484 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.137716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50560 (local address 127.7.170.1:46555)
0929 02:22:00.138292 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.138301 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.139275 (+ 974us) server_negotiation.cc:407] Connection header received
0929 02:22:00.139415 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.139422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.139632 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.139797 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:00.140393 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.141960 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.143843 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.145219 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.145335 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.146337 (+ 1002us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.148007 (+ 1670us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.148193 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":699,"threads_started":1}
I20260929 02:22:00.149937 23755 tablet_service.cc:1467] Processing CreateTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 (DEFAULT_TABLE table=test-table [id=a9161a995cae4155b0846a6b2ffdd64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:00.150241 23622 tablet_service.cc:1467] Processing CreateTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 (DEFAULT_TABLE table=test-table [id=a9161a995cae4155b0846a6b2ffdd64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:00.151575 23755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2074ee2c5c7244bfa7485d2826ff2df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:00.151577 23622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2074ee2c5c7244bfa7485d2826ff2df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:00.151595 23488 tablet_service.cc:1467] Processing CreateTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 (DEFAULT_TABLE table=test-table [id=a9161a995cae4155b0846a6b2ffdd64f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:00.152688 23488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2074ee2c5c7244bfa7485d2826ff2df8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:00.165205 23838 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Bootstrap starting.
I20260929 02:22:00.166742 23839 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Bootstrap starting.
I20260929 02:22:00.167183 23840 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Bootstrap starting.
I20260929 02:22:00.169044 23838 tablet_bootstrap.cc:654] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:00.170358 23838 log.cc:824] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:00.170595 23840 tablet_bootstrap.cc:654] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:00.171689 23840 log.cc:824] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:00.171865 23839 tablet_bootstrap.cc:654] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:00.173075 23839 log.cc:824] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:00.174217 23838 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: No bootstrap required, opened a new log
I20260929 02:22:00.174484 23838 ts_tablet_manager.cc:1397] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260929 02:22:00.175598 23840 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: No bootstrap required, opened a new log
I20260929 02:22:00.175932 23840 ts_tablet_manager.cc:1397] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 02:22:00.181619 23838 raft_consensus.cc:348] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.182099 23838 raft_consensus.cc:374] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:00.182200 23838 raft_consensus.cc:729] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25762067c8f447c9f0a589bc17b6886, State: Initialized, Role: FOLLOWER
I20260929 02:22:00.182874 23838 consensus_queue.cc:260] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.184702 23838 ts_tablet_manager.cc:1428] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260929 02:22:00.185550 23839 tablet_bootstrap.cc:492] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: No bootstrap required, opened a new log
I20260929 02:22:00.185861 23839 ts_tablet_manager.cc:1397] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.004s
I20260929 02:22:00.185981 23840 raft_consensus.cc:348] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.186328 23840 raft_consensus.cc:374] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:00.186426 23840 raft_consensus.cc:729] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428363a4d3cc4ce29df73a9fe0de72fc, State: Initialized, Role: FOLLOWER
I20260929 02:22:00.186955 23840 consensus_queue.cc:260] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.188175 23820 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:00.188586 23840 ts_tablet_manager.cc:1428] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260929 02:22:00.194427 23839 raft_consensus.cc:348] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.194818 23839 raft_consensus.cc:374] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:00.194916 23839 raft_consensus.cc:729] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f15783d7e9402da21d8c1367cb6c3b, State: Initialized, Role: FOLLOWER
I20260929 02:22:00.195422 23839 consensus_queue.cc:260] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.196336 23687 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:00.197018 23839 ts_tablet_manager.cc:1428] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260929 02:22:00.207016 23845 raft_consensus.cc:482] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:00.207284 23845 raft_consensus.cc:504] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.208981 23845 leader_election.cc:288] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
I20260929 02:22:00.215516 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.209462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:46101 (local address 127.7.170.1:46555)
0929 02:22:00.209704 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.209711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.209844 (+ 133us) server_negotiation.cc:407] Connection header received
0929 02:22:00.210028 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.210032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.210097 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.210183 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:00.210645 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.211661 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.213667 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.214664 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.214764 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.214839 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.215166 (+ 327us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.215259 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:22:00.215574 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.209395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.7.170.3:46101)
0929 02:22:00.209746 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.209764 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.209895 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.210211 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.210216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.210247 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:00.210498 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.210507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.211792 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.211797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.213500 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.213515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.214816 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.214822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.215018 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.215054 (+ 36us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.215151 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.215281 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":2,"spinlock_wait_cycles":25472}
I20260929 02:22:00.216082 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.209471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:45817 (local address 127.7.170.2:34507)
0929 02:22:00.209728 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.209734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.210286 (+ 552us) server_negotiation.cc:407] Connection header received
0929 02:22:00.210436 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.210440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.210507 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.210578 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:00.211152 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.212051 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.214262 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.215222 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.215338 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.215439 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.215778 (+ 339us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.215897 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260929 02:22:00.216557 23847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.209463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34507 (local address 127.7.170.3:45817)
0929 02:22:00.210191 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:00.210207 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:00.210332 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:00.210632 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:00.210636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:00.210662 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:00.210990 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.211000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.212213 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.212218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.214110 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:00.214125 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.215416 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.215422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:00.215595 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.215627 (+ 32us) client_negotiation.cc:769] Sending connection context
0929 02:22:00.216236 (+ 609us) client_negotiation.cc:240] Negotiation successful
0929 02:22:00.216332 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":434,"threads_started":1}
I20260929 02:22:00.216614 23508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2074ee2c5c7244bfa7485d2826ff2df8" candidate_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25762067c8f447c9f0a589bc17b6886" is_pre_election: true
I20260929 02:22:00.217204 23508 raft_consensus.cc:2454] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 428363a4d3cc4ce29df73a9fe0de72fc in term 0.
I20260929 02:22:00.217540 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2074ee2c5c7244bfa7485d2826ff2df8" candidate_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98f15783d7e9402da21d8c1367cb6c3b" is_pre_election: true
I20260929 02:22:00.217881 23710 leader_election.cc:302] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 428363a4d3cc4ce29df73a9fe0de72fc, f25762067c8f447c9f0a589bc17b6886; no voters:
I20260929 02:22:00.218020 23642 raft_consensus.cc:2454] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 428363a4d3cc4ce29df73a9fe0de72fc in term 0.
I20260929 02:22:00.218235 23845 raft_consensus.cc:2781] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:22:00.218348 23845 raft_consensus.cc:482] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:22:00.218472 23845 raft_consensus.cc:3037] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:00.220623 23845 raft_consensus.cc:504] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.221248 23845 leader_election.cc:288] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [CANDIDATE]: Term 1 election: Requested vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
I20260929 02:22:00.221691 23642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2074ee2c5c7244bfa7485d2826ff2df8" candidate_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98f15783d7e9402da21d8c1367cb6c3b"
I20260929 02:22:00.221750 23508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2074ee2c5c7244bfa7485d2826ff2df8" candidate_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25762067c8f447c9f0a589bc17b6886"
I20260929 02:22:00.221908 23642 raft_consensus.cc:3037] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:00.221937 23508 raft_consensus.cc:3037] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:00.225099 23508 raft_consensus.cc:2454] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 428363a4d3cc4ce29df73a9fe0de72fc in term 1.
I20260929 02:22:00.225098 23642 raft_consensus.cc:2454] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 428363a4d3cc4ce29df73a9fe0de72fc in term 1.
I20260929 02:22:00.225553 23710 leader_election.cc:302] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 428363a4d3cc4ce29df73a9fe0de72fc, f25762067c8f447c9f0a589bc17b6886; no voters:
I20260929 02:22:00.225863 23845 raft_consensus.cc:2781] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:22:00.226290 23845 raft_consensus.cc:686] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 LEADER]: Becoming Leader. State: Replica: 428363a4d3cc4ce29df73a9fe0de72fc, State: Running, Role: LEADER
I20260929 02:22:00.226835 23845 consensus_queue.cc:237] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:00.231638 23364 catalog_manager.cc:5495] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3). New cstate: current_term: 1 leader_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } health_report { overall_health: UNKNOWN } } }
I20260929 02:22:00.269627 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.264017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50010 (local address 127.7.170.3:33877)
0929 02:22:00.264228 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.264234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.264277 (+ 43us) server_negotiation.cc:407] Connection header received
0929 02:22:00.264513 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.264518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.264612 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.264746 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:22:00.265180 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.265933 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.267461 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.267859 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.267969 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.268498 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:22:00.269056 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:22:00.269186 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.269290 (+ 104us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.269381 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:22:00.278088 23508 raft_consensus.cc:1261] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Refusing update from remote peer 428363a4d3cc4ce29df73a9fe0de72fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:22:00.278088 23642 raft_consensus.cc:1261] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Refusing update from remote peer 428363a4d3cc4ce29df73a9fe0de72fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:22:00.279165 23845 consensus_queue.cc:1035] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:22:00.279757 23845 consensus_queue.cc:1035] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:22:00.293115 23852 mvcc.cc:205] Tried to move back new op lower bound from 7334496339048333312 to 7334496338853101568. Current Snapshot: MvccSnapshot[applied={T|T < 7334496339048333312}]
I20260929 02:22:00.303848 23363 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:22:00.304152 23363 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50854:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:22:00.307739 23363 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Sending DeleteTablet for 3 replicas of tablet 2074ee2c5c7244bfa7485d2826ff2df8
I20260929 02:22:00.308957 23755 tablet_service.cc:1514] Processing DeleteTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:00 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:49994
I20260929 02:22:00.309013 23622 tablet_service.cc:1514] Processing DeleteTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:00 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:53096
I20260929 02:22:00.309921 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23425
I20260929 02:22:00.309970 23488 tablet_service.cc:1514] Processing DeleteTablet for tablet 2074ee2c5c7244bfa7485d2826ff2df8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:00 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:50560
I20260929 02:22:00.316318 23861 tablet_replica.cc:330] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: stopping tablet replica
I20260929 02:22:00.317414 23861 raft_consensus.cc:2229] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:22:00.318666 23861 raft_consensus.cc:2258] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:22:00.322530 23861 ts_tablet_manager.cc:1905] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:22:00.324661 23862 tablet_replica.cc:330] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: stopping tablet replica
I20260929 02:22:00.327907 23860 tablet_replica.cc:330] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: stopping tablet replica
I20260929 02:22:00.328399 23860 raft_consensus.cc:2229] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:22:00.329146 23860 raft_consensus.cc:2258] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:22:00.329470 23862 raft_consensus.cc:2229] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:22:00.330075 23862 raft_consensus.cc:2258] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:22:00.331712 23861 ts_tablet_manager.cc:1918] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:22:00.331887 23861 log.cc:1196] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/2074ee2c5c7244bfa7485d2826ff2df8
I20260929 02:22:00.332365 23862 ts_tablet_manager.cc:1905] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:22:00.332775 23861 ts_tablet_manager.cc:1939] T 2074ee2c5c7244bfa7485d2826ff2df8 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting consensus metadata
I20260929 02:22:00.334743 23349 catalog_manager.cc:4841] TS 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): tablet 2074ee2c5c7244bfa7485d2826ff2df8 (table test-table [id=a9161a995cae4155b0846a6b2ffdd64f]) successfully deleted
I20260929 02:22:00.335421 23860 ts_tablet_manager.cc:1905] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:22:00.337950 23862 ts_tablet_manager.cc:1918] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:22:00.338107 23862 log.cc:1196] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/2074ee2c5c7244bfa7485d2826ff2df8
I20260929 02:22:00.338526 23862 ts_tablet_manager.cc:1939] T 2074ee2c5c7244bfa7485d2826ff2df8 P f25762067c8f447c9f0a589bc17b6886: Deleting consensus metadata
I20260929 02:22:00.340018 23350 catalog_manager.cc:4841] TS f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555): tablet 2074ee2c5c7244bfa7485d2826ff2df8 (table test-table [id=a9161a995cae4155b0846a6b2ffdd64f]) successfully deleted
Sep 29 02:22:00 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:22:00.342903 23860 ts_tablet_manager.cc:1918] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:22:00.343096 23860 log.cc:1196] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/2074ee2c5c7244bfa7485d2826ff2df8
I20260929 02:22:00.343993 23860 ts_tablet_manager.cc:1939] T 2074ee2c5c7244bfa7485d2826ff2df8 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting consensus metadata
I20260929 02:22:00.346107 23350 catalog_manager.cc:4841] TS 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): tablet 2074ee2c5c7244bfa7485d2826ff2df8 (table test-table [id=a9161a995cae4155b0846a6b2ffdd64f]) successfully deleted
I20260929 02:22:00.348846 23837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.311162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50574 (local address 127.7.170.1:46555)
0929 02:22:00.311335 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.311342 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.321358 (+ 10016us) server_negotiation.cc:407] Connection header received
0929 02:22:00.321870 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.321876 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.321965 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.322045 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:00.324108 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.325417 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.329544 (+ 4127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.330254 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.330396 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.342593 (+ 12197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:00.342607 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:00.342622 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:00.342664 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:00.345843 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.346890 (+ 1047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.346898 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.346901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.347039 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.347455 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.347458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.347460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.347811 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:00.348056 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.348423 (+ 367us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.348590 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:22:00.349807 23548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:00.446907 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23425
I20260929 02:22:00.468549 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23559
Sep 29 02:22:00 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:22:00.484323 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.469426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.7.170.2:34507)
0929 02:22:00.469612 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.469618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.469700 (+ 82us) server_negotiation.cc:407] Connection header received
0929 02:22:00.469880 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.469886 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.469963 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.470045 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:00.471298 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.472034 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.473636 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.474053 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.474165 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.478361 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:00.478376 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:00.478391 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:00.478426 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:00.481360 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.482020 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.482029 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.482031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.482154 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.482498 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.482502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.482504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.482879 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:00.483157 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.483606 (+ 449us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.483858 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260929 02:22:00.484985 23682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:00.590008 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23559
I20260929 02:22:00.603744 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23692
Sep 29 02:22:00 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648517, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:22:00.621449 23833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:00.604492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50026 (local address 127.7.170.3:33877)
0929 02:22:00.604691 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:00.604697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:00.604719 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:22:00.604795 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:00.604801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:00.604888 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:00.605017 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:00.606377 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.607619 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.609030 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:00.609700 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:00.609845 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:00.614120 (+ 4275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:00.614140 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:00.614162 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:00.614215 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:00.618177 (+ 3962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.619014 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.619024 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.619027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.619214 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:00.619613 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:00.619618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:00.619622 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:00.620128 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:00.620466 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:00.620738 (+ 272us) server_negotiation.cc:299] Negotiation successful
0929 02:22:00.620997 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260929 02:22:00.622242 23815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:00.746503 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23692
I20260929 02:22:00.764310 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23332
I20260929 02:22:00.765458 23394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:00.875368 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23332
I20260929 02:22:20.894784 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--webserver_interface=127.7.170.62
--webserver_port=44171
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:43475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:22:21.246649 23868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:21.246949 23868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:21.247066 23868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:21.254331 23868 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:22:21.254432 23868 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:22:21.254472 23868 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:22:21.254542 23868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:22:21.254619 23868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:21.254711 23868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:21.254760 23868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:22:21.254828 23868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:22:21.264757 23868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:43475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=44171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:21.266222 23868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:21.267609 23868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:21.289847 23875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:21.289921 23874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:21.289966 23877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:21.290745 23868 server_base.cc:1034] running on GCE node
I20260929 02:22:21.291620 23868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:21.293270 23868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:21.294536 23868 hybrid_clock.cc:648] HybridClock initialized: now 1790648541294457 us; error 51 us; skew 500 ppm
I20260929 02:22:21.294958 23868 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:21 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648541, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:21.306825 23868 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:22:21.308531 23868 webserver.cc:460] Webserver started at http://127.7.170.62:44171/ using document root <none> and password file <none>
I20260929 02:22:21.309422 23868 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:21.309547 23868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:21.315670 23868 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.002s
I20260929 02:22:21.318702 23884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:21.320076 23868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:22:21.320236 23868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "3fe8752ad98e43ceaeffae5099214cc5"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "7bf602c87a214fd0f40ad72356fe09f6"
server_key_iv: "7b283df76dfe85f74f5c53ab979926dd"
server_key_version: "encryptionkey@0"
I20260929 02:22:21.321349 23868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:21.338932 23868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:21.343266 23868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:21.343534 23868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:21.361434 23868 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:43475
I20260929 02:22:21.361457 23935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:43475 every 8 connection(s)
I20260929 02:22:21.363011 23868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:22:21.363931 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23868
I20260929 02:22:21.364882 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:46555
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=46311
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:22:21.370671 23936 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:22:21.374238 23936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap starting.
I20260929 02:22:21.401938 23936 log.cc:824] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:21.417239 23936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:22:21.417835 23936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap complete.
I20260929 02:22:21.431068 23936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:21.432291 23936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Initialized, Role: FOLLOWER
I20260929 02:22:21.433135 23936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:21.433447 23936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:22:21.433599 23936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:22:21.433804 23936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 1 FOLLOWER]: Advancing to term 2
I20260929 02:22:21.439929 23936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:21.440599 23936 leader_election.cc:302] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fe8752ad98e43ceaeffae5099214cc5; no voters:
I20260929 02:22:21.441315 23936 leader_election.cc:288] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260929 02:22:21.441460 23940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Leader election won for term 2
I20260929 02:22:21.442629 23940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 LEADER]: Becoming Leader. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Running, Role: LEADER
I20260929 02:22:21.443148 23940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:21.443714 23936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:22:21.445590 23941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:22:21.445566 23942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fe8752ad98e43ceaeffae5099214cc5. Latest consensus state: current_term: 2 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:22:21.445988 23941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:22:21.446084 23942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:22:21.446836 23945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:22:21.456445 23945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:22:21.457970 23945 catalog_manager.cc:1259] Loaded cluster ID: b5e46e07437f47a5a32aacc3930f22a9
I20260929 02:22:21.458137 23945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:22:21.462176 23945 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:22:21.463814 23945 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Loaded TSK: 0
I20260929 02:22:21.505386 23945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Generated new TSK 1
I20260929 02:22:21.505959 23945 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:22:21.742063 23938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:21.742367 23938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:21.742491 23938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:21.749730 23938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:21.749840 23938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:21.749974 23938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:22:21.759616 23938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:46555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=46311
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:21.760898 23938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:21.762210 23938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:21.778482 23964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:21.778543 23966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:21.779249 23938 server_base.cc:1034] running on GCE node
W20260929 02:22:21.781443 23963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:21.782121 23938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:21.783169 23938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:21.784399 23938 hybrid_clock.cc:648] HybridClock initialized: now 1790648541784338 us; error 61 us; skew 500 ppm
I20260929 02:22:21.784716 23938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:21 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648541, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:21.787822 23938 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:22:21.789359 23938 webserver.cc:460] Webserver started at http://127.7.170.1:46311/ using document root <none> and password file <none>
I20260929 02:22:21.790040 23938 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:21.790158 23938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:21.795799 23938 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:22:21.798828 23973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:21.800307 23938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:22:21.800477 23938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "f25762067c8f447c9f0a589bc17b6886"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "de7ca7749ad8cf4dc007eb3e0927e8c9"
server_key_iv: "f66687778646e273b065046e6aba9b36"
server_key_version: "encryptionkey@0"
I20260929 02:22:21.801483 23938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:21.829950 23938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:21.834465 23938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:21.834704 23938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:21.835639 23938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:21.837025 23938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:21.837110 23938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:21.837190 23938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:21.837241 23938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:21.867820 23938 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46555
I20260929 02:22:21.867875 24085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46555 every 8 connection(s)
I20260929 02:22:21.869271 23938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:22:21.874740 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 23938
I20260929 02:22:21.875466 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:34507
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=39015
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:22:21 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648541, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:22:21.893613 24089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:21.872357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:52031)
0929 02:22:21.873208 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:21.873258 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:21.875049 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:21.876825 (+ 1776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:21.876833 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:21.877488 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:21.878073 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:21.878091 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:21.880235 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:21.880239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:21.881525 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:21.881539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:21.882844 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:21.882851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:21.883057 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:21.883689 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:21.883711 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:21.886101 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:21.889347 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:21.889356 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:21.889371 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:21.889794 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:21.890217 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:21.890223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:21.890227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:21.890409 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:21.891149 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:21.891156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:21.891475 (+ 319us) client_negotiation.cc:769] Sending connection context
0929 02:22:21.891718 (+ 243us) client_negotiation.cc:240] Negotiation successful
0929 02:22:21.892274 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":318,"threads_started":1}
I20260929 02:22:21.893615 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:21.872158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:52031 (local address 127.7.170.62:43475)
0929 02:22:21.872712 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:21.872723 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:21.873428 (+ 705us) server_negotiation.cc:407] Connection header received
0929 02:22:21.875690 (+ 2262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:21.875710 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:21.876195 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:21.876624 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:21.878279 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:21.880056 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:21.881670 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:21.882624 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:21.882789 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:21.886375 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:21.886383 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:21.886398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:21.886444 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:21.889210 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:21.889931 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:21.889938 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:21.889944 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:21.890112 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:21.890522 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:21.890525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:21.890527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:21.891002 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:21.891185 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:21.891964 (+ 779us) server_negotiation.cc:299] Negotiation successful
0929 02:22:21.892347 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":235,"threads_started":1}
I20260929 02:22:21.896679 24086 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:22:21.897054 24086 heartbeater.cc:464] Registering TS with master...
I20260929 02:22:21.897956 24086 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:22:21.900187 23901 ts_manager.cc:194] Registered new tserver with Master: f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
I20260929 02:22:21.901680 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:52031
W20260929 02:22:22.200119 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:22.200376 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:22.200502 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:22.207692 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:22.207798 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:22.207935 24090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:22:22.217376 24090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=39015
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:22.218608 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:22.219887 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:22.232041 24098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:22.232110 24096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:22.232273 24095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:22.233068 24090 server_base.cc:1034] running on GCE node
I20260929 02:22:22.233608 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:22.234467 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:22.235666 24090 hybrid_clock.cc:648] HybridClock initialized: now 1790648542235619 us; error 37 us; skew 500 ppm
I20260929 02:22:22.235980 24090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:22 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:22.238873 24090 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:22:22.240224 24090 webserver.cc:460] Webserver started at http://127.7.170.2:39015/ using document root <none> and password file <none>
I20260929 02:22:22.240881 24090 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:22.241035 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:22.246294 24090 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260929 02:22:22.248776 24105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.250075 24090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:22:22.250226 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "98f15783d7e9402da21d8c1367cb6c3b"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "e2e511216eec6a59d415ff9b66d9e139"
server_key_iv: "e7ec82fc44811f4047217bc31b0f6562"
server_key_version: "encryptionkey@0"
I20260929 02:22:22.251217 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:22.261658 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:22.265812 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:22.266065 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:22.267030 24090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:22.268309 24090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:22.268402 24090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.268514 24090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:22.268584 24090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.299005 24090 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34507
I20260929 02:22:22.299079 24217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34507 every 8 connection(s)
I20260929 02:22:22.300491 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:22:22.310608 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24090
I20260929 02:22:22.311357 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:33877
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=41579
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 29 02:22:22 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:22:22.319813 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.302926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:54885 (local address 127.7.170.62:43475)
0929 02:22:22.303131 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.303137 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.304273 (+ 1136us) server_negotiation.cc:407] Connection header received
0929 02:22:22.306126 (+ 1853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.306131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.306201 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.306290 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:22.307668 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.308367 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.309898 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.310428 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.310525 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.314590 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:22.314595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:22.314599 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:22.314633 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:22.317007 (+ 2374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.317754 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.317757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.317759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.317877 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.318299 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.318302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.318304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.318558 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:22.318663 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.319413 (+ 750us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.319578 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:22:22.320847 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.303442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:54885)
0929 02:22:22.304122 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:22.304158 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:22.305886 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:22.306487 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:22.306495 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:22.306983 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:22.307525 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.307543 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.308496 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.308500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.309770 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.309781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.310645 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.310653 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.310950 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.311695 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:22.311722 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:22.314359 (+ 2637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:22.317136 (+ 2777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:22.317144 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:22.317160 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:22.317659 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:22.317983 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:22.317987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:22.317990 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:22.318211 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:22.318672 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:22.318680 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:22.319072 (+ 392us) client_negotiation.cc:769] Sending connection context
0929 02:22:22.319351 (+ 279us) client_negotiation.cc:240] Negotiation successful
0929 02:22:22.319849 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":255,"threads_started":1}
I20260929 02:22:22.322324 24218 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:22:22.322609 24218 heartbeater.cc:464] Registering TS with master...
I20260929 02:22:22.323241 24218 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:22:22.324622 23901 ts_manager.cc:194] Registered new tserver with Master: 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507)
I20260929 02:22:22.325774 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:54885
W20260929 02:22:22.633671 24221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:22.633934 24221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:22.634033 24221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:22.641018 24221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:22.641116 24221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:22.641235 24221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:22:22.650822 24221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:33877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=41579
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:22.652047 24221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:22.653321 24221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:22.666507 24229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:22.666543 24227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:22.666570 24226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:22.667686 24221 server_base.cc:1034] running on GCE node
I20260929 02:22:22.668285 24221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:22.669294 24221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:22.670500 24221 hybrid_clock.cc:648] HybridClock initialized: now 1790648542670445 us; error 49 us; skew 500 ppm
I20260929 02:22:22.670809 24221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:22 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:22.673930 24221 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:22:22.675516 24221 webserver.cc:460] Webserver started at http://127.7.170.3:41579/ using document root <none> and password file <none>
I20260929 02:22:22.676239 24221 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:22.676376 24221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:22.681926 24221 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260929 02:22:22.684654 24236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.686061 24221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:22:22.686204 24221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "b8bb5ec174ca504aa57c4588216973bf"
server_key_iv: "12c483cf3884e0ff071c0bfa333629ce"
server_key_version: "encryptionkey@0"
I20260929 02:22:22.687158 24221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:22.703691 24221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:22.708158 24221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:22.708423 24221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:22.709416 24221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:22.710724 24221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:22.710825 24221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.710923 24221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:22.710992 24221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:22.745571 24221 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33877
I20260929 02:22:22.745632 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33877 every 8 connection(s)
I20260929 02:22:22.746975 24221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:22:22.757411 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24221
Sep 29 02:22:22 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648542, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:22:22.767606 24088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.749533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:56069 (local address 127.7.170.62:43475)
0929 02:22:22.749727 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.749732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.750882 (+ 1150us) server_negotiation.cc:407] Connection header received
0929 02:22:22.752651 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.752657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.752736 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.752832 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:22.754369 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.755084 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.756582 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.757070 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.757173 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.761322 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:22.761329 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:22.761342 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:22.761386 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:22.764278 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.765192 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.765198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.765201 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.765368 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.765902 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.765930 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.765932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.766193 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:22.766307 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.767189 (+ 882us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.767351 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:22:22.768795 24351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.750003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:56069)
0929 02:22:22.750687 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:22.750724 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:22.752387 (+ 1663us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:22.753092 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:22.753100 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:22.753634 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:22.754187 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.754203 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.755230 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.755234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.756451 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.756463 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.757424 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.757429 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.757671 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.758751 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:22.758782 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:22.761127 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:22.764440 (+ 3313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:22.764450 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:22.764468 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:22.765076 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:22.765505 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:22.765510 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:22.765513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:22.765760 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:22.766334 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:22.766343 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:22.766790 (+ 447us) client_negotiation.cc:769] Sending connection context
0929 02:22:22.767108 (+ 318us) client_negotiation.cc:240] Negotiation successful
0929 02:22:22.767665 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":274,"threads_started":1}
I20260929 02:22:22.770263 24349 heartbeater.cc:347] Connected to a master server at 127.7.170.62:43475
I20260929 02:22:22.770545 24349 heartbeater.cc:464] Registering TS with master...
I20260929 02:22:22.771433 24349 heartbeater.cc:511] Master 127.7.170.62:43475 requested a full tablet report, sending...
I20260929 02:22:22.773144 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.758281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.7.170.62:43475)
0929 02:22:22.758777 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.758782 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.758988 (+ 206us) server_negotiation.cc:407] Connection header received
0929 02:22:22.759149 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.759153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.759208 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.759290 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:22.761172 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.762246 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.763739 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.764401 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.764500 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.767899 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:22.767913 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:22.767917 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:22.767945 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:22.770548 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.771299 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.771302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.771304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.771407 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.771798 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.771801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.771803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.772056 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:22.772249 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.772553 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.772722 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":241,"threads_started":1}
I20260929 02:22:22.773120 23901 ts_manager.cc:194] Registered new tserver with Master: 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
I20260929 02:22:22.774104 23901 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:56069
I20260929 02:22:22.775584 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:22:22.781641 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.776477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48462 (local address 127.7.170.62:43475)
0929 02:22:22.776627 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.776631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.777167 (+ 536us) server_negotiation.cc:407] Connection header received
0929 02:22:22.777397 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.777401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.777542 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.777646 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:22:22.778037 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.778686 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.780083 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.780472 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.780560 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.781005 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:22:22.781198 (+ 193us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0929 02:22:22.781449 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48462: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260929 02:22:22.781754 24352 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48462: token expired
W20260929 02:22:22.782342 23826 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.170.62:43475)
I20260929 02:22:22.794157 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.782777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48470 (local address 127.7.170.62:43475)
0929 02:22:22.782957 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.782961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.782974 (+ 13us) server_negotiation.cc:407] Connection header received
0929 02:22:22.783008 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.783011 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.783058 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.783131 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:22.784378 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.785132 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.786478 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.786825 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.786909 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.789475 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:22.789480 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:22.789483 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:22.789513 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:22.791950 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.792561 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.792565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.792567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.792682 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:22.793038 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:22.793041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:22.793043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:22.793337 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:22.793513 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.793777 (+ 264us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.793930 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260929 02:22:22.797693 23900 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:22:22.804759 23900 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:22:22.826508 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.816509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.7.170.3:33877)
0929 02:22:22.817225 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.817233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.817836 (+ 603us) server_negotiation.cc:407] Connection header received
0929 02:22:22.818137 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.818145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.818380 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.818544 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:22.819612 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.821196 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.823826 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.825002 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.825120 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.825338 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.825885 (+ 547us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.826060 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":422,"threads_started":1}
I20260929 02:22:22.826809 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.816040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35730 (local address 127.7.170.2:34507)
0929 02:22:22.816864 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.816873 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.817907 (+ 1034us) server_negotiation.cc:407] Connection header received
0929 02:22:22.818142 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.818149 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.818395 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.818546 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:22.820073 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.821446 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.823570 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.825005 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.825642 (+ 637us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.825853 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.826363 (+ 510us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.826517 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":532,"threads_started":1}
I20260929 02:22:22.828029 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.818278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55276 (local address 127.7.170.1:46555)
0929 02:22:22.819323 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:22.819332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:22.819863 (+ 531us) server_negotiation.cc:407] Connection header received
0929 02:22:22.820118 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:22.820125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:22.820347 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:22.820512 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:22.821304 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.822790 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.825186 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.826573 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.826688 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.827480 (+ 792us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:22.827546 (+ 66us) server_negotiation.cc:299] Negotiation successful
0929 02:22:22.827702 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":894,"thread_start_us":748,"threads_started":1}
I20260929 02:22:22.828061 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.816584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33877 (local address 127.0.0.1:48434)
0929 02:22:22.817667 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:22.817737 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:22.817925 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:22.818862 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:22.818869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:22.818918 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:22.819371 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.819388 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.821382 (+ 1994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.821387 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.823642 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.823655 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.825334 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.825342 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.825586 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.825636 (+ 50us) client_negotiation.cc:769] Sending connection context
0929 02:22:22.826793 (+ 1157us) client_negotiation.cc:240] Negotiation successful
0929 02:22:22.827066 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":947,"mutex_wait_us":55,"spinlock_wait_cycles":1280,"thread_start_us":937,"threads_started":1}
I20260929 02:22:22.828518 24359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.815795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34507 (local address 127.0.0.1:35730)
0929 02:22:22.817667 (+ 1872us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:22.817731 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:22.817901 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:22.818862 (+ 961us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:22.818869 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:22.818909 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:22.819903 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.819915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.821600 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.821605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.823373 (+ 1768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.823387 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.825996 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.826002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.826168 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.826198 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:22:22.827057 (+ 859us) client_negotiation.cc:240] Negotiation successful
0929 02:22:22.827181 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1606,"mutex_wait_us":639,"spinlock_wait_cycles":896,"thread_start_us":1446,"threads_started":1}
I20260929 02:22:22.828857 24363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:22.818066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55276)
0929 02:22:22.819760 (+ 1694us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:22.819778 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:22.819958 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:22.820710 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:22.820714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:22.820739 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:22.821120 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.821133 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.822940 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.822945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.825029 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:22.825042 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:22.826895 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:22.826901 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:22.827081 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:22.827111 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:22:22.827180 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:22:22.827291 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1576,"mutex_wait_us":3,"thread_start_us":1425,"threads_started":1}
I20260929 02:22:22.831624 24284 tablet_service.cc:1467] Processing CreateTablet for tablet cd11c382a01244049d32b4d244e32911 (DEFAULT_TABLE table=test-table [id=6d34f89011974515ae2cd0461505f2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:22.831820 24153 tablet_service.cc:1467] Processing CreateTablet for tablet cd11c382a01244049d32b4d244e32911 (DEFAULT_TABLE table=test-table [id=6d34f89011974515ae2cd0461505f2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:22.831908 24021 tablet_service.cc:1467] Processing CreateTablet for tablet cd11c382a01244049d32b4d244e32911 (DEFAULT_TABLE table=test-table [id=6d34f89011974515ae2cd0461505f2d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:22:22.833151 24153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11c382a01244049d32b4d244e32911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:22.833163 24021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11c382a01244049d32b4d244e32911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:22.833159 24284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd11c382a01244049d32b4d244e32911. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:22:22.844619 24365 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Bootstrap starting.
I20260929 02:22:22.847875 24366 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Bootstrap starting.
I20260929 02:22:22.849192 24367 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Bootstrap starting.
I20260929 02:22:22.849699 24365 tablet_bootstrap.cc:654] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:22.851013 24365 log.cc:824] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:22.851256 24366 tablet_bootstrap.cc:654] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:22.851917 24367 tablet_bootstrap.cc:654] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Neither blocks nor log segments found. Creating new log.
I20260929 02:22:22.852295 24366 log.cc:824] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:22.853005 24367 log.cc:824] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:22.854595 24365 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: No bootstrap required, opened a new log
I20260929 02:22:22.854866 24365 ts_tablet_manager.cc:1397] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260929 02:22:22.855213 24366 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: No bootstrap required, opened a new log
I20260929 02:22:22.855466 24366 ts_tablet_manager.cc:1397] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260929 02:22:22.856369 24367 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: No bootstrap required, opened a new log
I20260929 02:22:22.856580 24367 ts_tablet_manager.cc:1397] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260929 02:22:22.863705 24367 raft_consensus.cc:348] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.864250 24367 raft_consensus.cc:374] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:22.864621 24367 raft_consensus.cc:729] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f15783d7e9402da21d8c1367cb6c3b, State: Initialized, Role: FOLLOWER
I20260929 02:22:22.865355 24367 consensus_queue.cc:260] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.865379 24365 raft_consensus.cc:348] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.865378 24366 raft_consensus.cc:348] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.865840 24366 raft_consensus.cc:374] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:22.865841 24365 raft_consensus.cc:374] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:22:22.865947 24365 raft_consensus.cc:729] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428363a4d3cc4ce29df73a9fe0de72fc, State: Initialized, Role: FOLLOWER
I20260929 02:22:22.865947 24366 raft_consensus.cc:729] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25762067c8f447c9f0a589bc17b6886, State: Initialized, Role: FOLLOWER
I20260929 02:22:22.866554 24366 consensus_queue.cc:260] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.866554 24365 consensus_queue.cc:260] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:22.867276 24218 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:22.868032 24367 ts_tablet_manager.cc:1428] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260929 02:22:22.874732 24365 ts_tablet_manager.cc:1428] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Time spent starting tablet: real 0.020s user 0.012s sys 0.007s
I20260929 02:22:22.875557 24086 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:22.875586 24349 heartbeater.cc:503] Master 127.7.170.62:43475 was elected leader, sending a full tablet report...
I20260929 02:22:22.876264 24366 ts_tablet_manager.cc:1428] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Time spent starting tablet: real 0.021s user 0.018s sys 0.003s
W20260929 02:22:22.877020 24087 tablet.cc:2369] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:22:22.998523 24350 tablet.cc:2369] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:22:23.044540 24371 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:23.044857 24371 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:23.046566 24371 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877), f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
W20260929 02:22:23.052393 24219 tablet.cc:2369] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:22:23.054180 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.046972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:47151 (local address 127.7.170.3:33877)
0929 02:22:23.047154 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.047161 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.047713 (+ 552us) server_negotiation.cc:407] Connection header received
0929 02:22:23.047954 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.047960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.048046 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.048114 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:23.049143 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.050151 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.052084 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.053251 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.053381 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.053480 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.053827 (+ 347us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.053948 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:22:23.054200 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.047942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:48751 (local address 127.7.170.1:46555)
0929 02:22:23.048163 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.048169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.048252 (+ 83us) server_negotiation.cc:407] Connection header received
0929 02:22:23.048383 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.048387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.048455 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.048549 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:22:23.049241 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.050178 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.052130 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.053233 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.053349 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.053454 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.053848 (+ 394us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.053967 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260929 02:22:23.054366 24374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.046935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33877 (local address 127.7.170.2:47151)
0929 02:22:23.047568 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.047591 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:23.047757 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:23.048340 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:23.048345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:23.048373 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:23.048879 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:23.048909 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.050284 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.050290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:23.051942 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:23.051956 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.053487 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.053500 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:23.053681 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.053704 (+ 23us) client_negotiation.cc:769] Sending connection context
0929 02:22:23.053785 (+ 81us) client_negotiation.cc:240] Negotiation successful
0929 02:22:23.053920 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":104,"thread_start_us":335,"threads_started":1}
I20260929 02:22:23.054651 24375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.047634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.7.170.2:48751)
0929 02:22:23.048163 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.048177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:23.048279 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:23.048568 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:23.048571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:23.048589 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:22:23.049100 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:23.049111 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.050296 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.050302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:23.051960 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:23.051973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.053455 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.053461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:23.053626 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.053690 (+ 64us) client_negotiation.cc:769] Sending connection context
0929 02:22:23.053774 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:22:23.053907 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":72,"thread_start_us":254,"threads_started":1}
I20260929 02:22:23.055285 24304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11c382a01244049d32b4d244e32911" candidate_uuid: "98f15783d7e9402da21d8c1367cb6c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" is_pre_election: true
I20260929 02:22:23.055624 24041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11c382a01244049d32b4d244e32911" candidate_uuid: "98f15783d7e9402da21d8c1367cb6c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25762067c8f447c9f0a589bc17b6886" is_pre_election: true
I20260929 02:22:23.055897 24304 raft_consensus.cc:2454] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f15783d7e9402da21d8c1367cb6c3b in term 0.
I20260929 02:22:23.056180 24041 raft_consensus.cc:2454] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 98f15783d7e9402da21d8c1367cb6c3b in term 0.
I20260929 02:22:23.056653 24108 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b; no voters:
I20260929 02:22:23.057072 24371 raft_consensus.cc:2781] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:22:23.057219 24371 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:22:23.057338 24371 raft_consensus.cc:3037] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:23.059443 24371 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:23.060014 24371 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [CANDIDATE]: Term 1 election: Requested vote from peers 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877), f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555)
I20260929 02:22:23.060539 24304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11c382a01244049d32b4d244e32911" candidate_uuid: "98f15783d7e9402da21d8c1367cb6c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
I20260929 02:22:23.060563 24041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd11c382a01244049d32b4d244e32911" candidate_uuid: "98f15783d7e9402da21d8c1367cb6c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f25762067c8f447c9f0a589bc17b6886"
I20260929 02:22:23.060771 24041 raft_consensus.cc:3037] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:23.060771 24304 raft_consensus.cc:3037] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:22:23.063972 24304 raft_consensus.cc:2454] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f15783d7e9402da21d8c1367cb6c3b in term 1.
I20260929 02:22:23.063972 24041 raft_consensus.cc:2454] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 98f15783d7e9402da21d8c1367cb6c3b in term 1.
I20260929 02:22:23.064409 24108 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98f15783d7e9402da21d8c1367cb6c3b, f25762067c8f447c9f0a589bc17b6886; no voters:
I20260929 02:22:23.064685 24371 raft_consensus.cc:2781] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:22:23.065209 24371 raft_consensus.cc:686] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 LEADER]: Becoming Leader. State: Replica: 98f15783d7e9402da21d8c1367cb6c3b, State: Running, Role: LEADER
I20260929 02:22:23.065686 24371 consensus_queue.cc:237] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:23.070441 23900 catalog_manager.cc:5495] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2). New cstate: current_term: 1 leader_uuid: "98f15783d7e9402da21d8c1367cb6c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } health_report { overall_health: UNKNOWN } } }
I20260929 02:22:23.121644 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.116586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.7.170.2:34507)
0929 02:22:23.116770 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.116775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.116792 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:22:23.116879 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.116882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.116997 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.117113 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:22:23.117511 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.118224 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.119534 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.119936 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.120032 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.120512 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:22:23.121093 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:22:23.121202 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.121299 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.121365 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:22:23.129773 24041 raft_consensus.cc:1261] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Refusing update from remote peer 98f15783d7e9402da21d8c1367cb6c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:22:23.129773 24304 raft_consensus.cc:1261] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Refusing update from remote peer 98f15783d7e9402da21d8c1367cb6c3b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:22:23.130745 24376 consensus_queue.cc:1035] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:22:23.131100 24371 consensus_queue.cc:1035] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:22:23.161513 23901 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:22:23.161783 23901 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:22:23.165431 23901 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Sending DeleteTablet for 3 replicas of tablet cd11c382a01244049d32b4d244e32911
I20260929 02:22:23.166699 24153 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11c382a01244049d32b4d244e32911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:23 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:35730
I20260929 02:22:23.166754 24284 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11c382a01244049d32b4d244e32911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:23 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:48434
I20260929 02:22:23.166813 24021 tablet_service.cc:1514] Processing DeleteTablet for tablet cd11c382a01244049d32b4d244e32911 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:22:23 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:55276
I20260929 02:22:23.168013 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23938
I20260929 02:22:23.198969 24364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.168903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55290 (local address 127.7.170.1:46555)
0929 02:22:23.169556 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.169563 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.169582 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:22:23.169646 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.169651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.169765 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.169961 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:23.172078 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.173311 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.175589 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.176354 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.176482 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.184793 (+ 8311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:23.184818 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:23.184841 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:23.184909 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:23.190732 (+ 5823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.194158 (+ 3426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.194171 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.194176 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.194420 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.195139 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.195145 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.195149 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.195958 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:23.196305 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.198213 (+ 1908us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.198504 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505}
I20260929 02:22:23.199784 24081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:23.217195 24391 tablet_replica.cc:330] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: stopping tablet replica
I20260929 02:22:23.217308 24390 tablet_replica.cc:330] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: stopping tablet replica
I20260929 02:22:23.217708 24390 raft_consensus.cc:2229] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:22:23.217705 24391 raft_consensus.cc:2229] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:22:23.218237 24391 raft_consensus.cc:2258] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:22:23.218392 24390 raft_consensus.cc:2258] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:22:23.220482 24391 ts_tablet_manager.cc:1905] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:22:23.220489 24390 ts_tablet_manager.cc:1905] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:22:23.227336 24391 ts_tablet_manager.cc:1918] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:22:23.227511 24391 log.cc:1196] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/cd11c382a01244049d32b4d244e32911
I20260929 02:22:23.227605 24390 ts_tablet_manager.cc:1918] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:22:23.227753 24390 log.cc:1196] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/cd11c382a01244049d32b4d244e32911
I20260929 02:22:23.228026 24391 ts_tablet_manager.cc:1939] T cd11c382a01244049d32b4d244e32911 P 428363a4d3cc4ce29df73a9fe0de72fc: Deleting consensus metadata
I20260929 02:22:23.228291 24390 ts_tablet_manager.cc:1939] T cd11c382a01244049d32b4d244e32911 P 98f15783d7e9402da21d8c1367cb6c3b: Deleting consensus metadata
I20260929 02:22:23.230137 23887 catalog_manager.cc:4841] TS 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): tablet cd11c382a01244049d32b4d244e32911 (table test-table [id=6d34f89011974515ae2cd0461505f2d7]) successfully deleted
I20260929 02:22:23.230252 23886 catalog_manager.cc:4841] TS 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): tablet cd11c382a01244049d32b4d244e32911 (table test-table [id=6d34f89011974515ae2cd0461505f2d7]) successfully deleted
I20260929 02:22:23.364481 24389 tablet_replica.cc:330] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: stopping tablet replica
I20260929 02:22:23.364679 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23938
W20260929 02:22:23.385938 23887 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.170.1:46555 (error 108)
I20260929 02:22:23.386220 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24090
I20260929 02:22:23.388638 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.387748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55304)
0929 02:22:23.388300 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.388433 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":342,"threads_started":1}
W20260929 02:22:23.388972 23887 catalog_manager.cc:4584] TS f25762067c8f447c9f0a589bc17b6886 (127.7.170.1:46555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd11c382a01244049d32b4d244e32911: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
I20260929 02:22:23.403187 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.387362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.7.170.2:34507)
0929 02:22:23.387548 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.387554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.387575 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:22:23.387643 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.387648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.387733 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.387841 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:23.389410 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.390616 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.392155 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.392875 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.393051 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.395764 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:23.395783 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:23.395804 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:23.395866 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:23.399869 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.400684 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.400695 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.400699 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.400885 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.401452 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.401457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.401461 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.401936 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:23.402264 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.402558 (+ 294us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.402790 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:22:23.404008 24213 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:23.442795 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.442242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55320)
0929 02:22:23.442492 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.442613 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 02:22:23.443898 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.443476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55330)
0929 02:22:23.443640 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.443727 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260929 02:22:23.504365 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.503821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55334)
0929 02:22:23.504064 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.504176 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 02:22:23.505499 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.505089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55338)
0929 02:22:23.505244 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.505331 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:22:23.518139 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24090
I20260929 02:22:23.533288 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24221
I20260929 02:22:23.546665 24361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.534055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.7.170.3:33877)
0929 02:22:23.534237 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:23.534242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:23.534263 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:22:23.534403 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:23.534421 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:23.534490 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:23.534573 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:23.535965 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.536673 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.538164 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:23.538593 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:23.538712 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:23.541208 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:23.541223 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:23.541239 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:23.541274 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:23.543838 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.544552 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.544562 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.544564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.544707 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:23.545148 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:23.545154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:23.545157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:23.545536 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:23.545840 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:23.546129 (+ 289us) server_negotiation.cc:299] Negotiation successful
0929 02:22:23.546353 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:22:23.547299 24344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:23.586231 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.585651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55354)
0929 02:22:23.585918 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.586041 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260929 02:22:23.587424 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.586976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55366)
0929 02:22:23.587201 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.587276 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260929 02:22:23.635902 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24221
I20260929 02:22:23.650164 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 23868
I20260929 02:22:23.651257 23931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:22:23.762856 24394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:23.762367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46555 (local address 127.0.0.1:55382)
0929 02:22:23.762599 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:23.762704 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 02:22:23.762961 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 23868
I20260929 02:22:43.781920 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--webserver_interface=127.7.170.62
--webserver_port=44171
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:43475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:22:44.154544 24398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:44.154915 24398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:44.155064 24398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:44.162499 24398 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260929 02:22:44.162617 24398 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260929 02:22:44.162686 24398 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260929 02:22:44.162760 24398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:22:44.162837 24398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:44.162937 24398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:44.162997 24398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:22:44.163072 24398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:22:44.173394 24398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:43475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:43475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=44171
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:44.175024 24398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:44.176580 24398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:44.190037 24403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:44.190078 24404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:44.190119 24406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:44.190866 24398 server_base.cc:1034] running on GCE node
I20260929 02:22:44.191731 24398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:44.193449 24398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:44.194676 24398 hybrid_clock.cc:648] HybridClock initialized: now 1790648564194600 us; error 63 us; skew 500 ppm
I20260929 02:22:44.195062 24398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:44 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648564, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:44.205874 24398 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:22:44.207450 24398 webserver.cc:460] Webserver started at http://127.7.170.62:44171/ using document root <none> and password file <none>
I20260929 02:22:44.208231 24398 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:44.208318 24398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:44.214429 24398 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:22:44.217542 24413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:44.218986 24398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:22:44.219137 24398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "3fe8752ad98e43ceaeffae5099214cc5"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "7bf602c87a214fd0f40ad72356fe09f6"
server_key_iv: "7b283df76dfe85f74f5c53ab979926dd"
server_key_version: "encryptionkey@0"
I20260929 02:22:44.220230 24398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:44.233465 24398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:44.238198 24398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:44.238462 24398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:44.257966 24398 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:43475
I20260929 02:22:44.257984 24464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:43475 every 8 connection(s)
I20260929 02:22:44.259673 24398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:22:44.261348 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24398
I20260929 02:22:44.262477 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:46555
--local_ip_for_outbound_sockets=127.7.170.1
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=46311
--webserver_interface=127.7.170.1
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:22:44.270027 24465 sys_catalog.cc:263] Verifying existing consensus state
I20260929 02:22:44.274801 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap starting.
I20260929 02:22:44.299007 24465 log.cc:824] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:44.316443 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:22:44.317054 24465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Bootstrap complete.
I20260929 02:22:44.327224 24465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:44.328209 24465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Initialized, Role: FOLLOWER
I20260929 02:22:44.328913 24465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:44.329178 24465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:22:44.329294 24465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:22:44.329463 24465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 2 FOLLOWER]: Advancing to term 3
I20260929 02:22:44.333575 24465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:44.334095 24465 leader_election.cc:302] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fe8752ad98e43ceaeffae5099214cc5; no voters:
I20260929 02:22:44.334769 24465 leader_election.cc:288] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260929 02:22:44.334936 24469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 3 FOLLOWER]: Leader election won for term 3
I20260929 02:22:44.336076 24469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [term 3 LEADER]: Becoming Leader. State: Replica: 3fe8752ad98e43ceaeffae5099214cc5, State: Running, Role: LEADER
I20260929 02:22:44.336737 24469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } }
I20260929 02:22:44.337167 24465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:22:44.339742 24471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fe8752ad98e43ceaeffae5099214cc5. Latest consensus state: current_term: 3 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:22:44.339779 24470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3fe8752ad98e43ceaeffae5099214cc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fe8752ad98e43ceaeffae5099214cc5" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 43475 } } }
I20260929 02:22:44.340161 24470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:22:44.340167 24471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:22:44.341327 24477 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:22:44.361310 24477 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:22:44.362963 24477 catalog_manager.cc:1259] Loaded cluster ID: b5e46e07437f47a5a32aacc3930f22a9
I20260929 02:22:44.363168 24477 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:22:44.367522 24477 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:22:44.369285 24477 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Loaded TSK: 0
I20260929 02:22:44.369531 24477 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3fe8752ad98e43ceaeffae5099214cc5: Loaded TSK: 1
F20260929 02:22:44.370915 24477 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5f4223a9ad google::LogMessage::Fail() at ??:0
@ 0x7f5f4223d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f5f4223a4de google::LogMessage::Flush() at ??:0
@ 0x7f5f4223ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5f45f9e7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f5f45f8d73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f5f57c691c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f5f57c68783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f5f57cf42a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f5f57cf40c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f5f57d3e5e9 std::function<>::operator()() at ??:0
@ 0x7f5f57c6ccdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f5f57c6ac36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5f57cf2610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f5f57cf2439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5f44e86345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5f44e8991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f5f44e89799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f5f44e64eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5f4813e6db start_thread at ??:0
@ 0x7f5f410d461f clone at ??:0
W20260929 02:22:44.736899 24467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:44.737286 24467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:44.737448 24467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:44.746044 24467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:44.746158 24467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:44.746279 24467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:22:44.756191 24467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:46555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=46311
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:44.757688 24467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:44.759091 24467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:44.772341 24515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:44.772365 24513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:44.772576 24512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:44.773180 24467 server_base.cc:1034] running on GCE node
I20260929 02:22:44.773741 24467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:44.774686 24467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:44.775899 24467 hybrid_clock.cc:648] HybridClock initialized: now 1790648564775851 us; error 40 us; skew 500 ppm
I20260929 02:22:44.776192 24467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:44 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648564, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:44.779253 24467 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:22:44.780737 24467 webserver.cc:460] Webserver started at http://127.7.170.1:46311/ using document root <none> and password file <none>
I20260929 02:22:44.781495 24467 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:44.781612 24467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:44.787317 24467 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:22:44.790393 24522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:44.791765 24467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:22:44.791940 24467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "f25762067c8f447c9f0a589bc17b6886"
format_stamp: "Formatted at 2026-09-29 02:21:58 on dist-test-slave-8cs2"
server_key: "de7ca7749ad8cf4dc007eb3e0927e8c9"
server_key_iv: "f66687778646e273b065046e6aba9b36"
server_key_version: "encryptionkey@0"
I20260929 02:22:44.793145 24467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:44.828747 24467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:44.832950 24467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:44.833204 24467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:44.834213 24467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:44.836112 24529 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260929 02:22:44.842196 24467 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260929 02:22:44.842325 24467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20260929 02:22:44.842450 24467 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260929 02:22:44.845781 24467 ts_tablet_manager.cc:610] Registered 1 tablets
I20260929 02:22:44.845880 24467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260929 02:22:44.846244 24529 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Bootstrap starting.
I20260929 02:22:44.872664 24529 log.cc:824] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Log is configured to *not* fsync() on all Append() calls
I20260929 02:22:44.885500 24529 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260929 02:22:44.886173 24529 tablet_bootstrap.cc:492] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Bootstrap complete.
I20260929 02:22:44.886608 24467 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46555
I20260929 02:22:44.886646 24636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46555 every 8 connection(s)
I20260929 02:22:44.887441 24529 ts_tablet_manager.cc:1397] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Time spent bootstrapping tablet: real 0.042s user 0.023s sys 0.015s
I20260929 02:22:44.887871 24467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:22:44.889489 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24467
I20260929 02:22:44.890265 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:34507
--local_ip_for_outbound_sockets=127.7.170.2
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=39015
--webserver_interface=127.7.170.2
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:22:44.893062 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:44.890841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:48749)
0929 02:22:44.891755 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:44.892032 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":397,"threads_started":1}
W20260929 02:22:44.893699 24637 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:43475 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:43475: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
I20260929 02:22:44.894510 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:44.894185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:36015)
0929 02:22:44.894333 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:44.894402 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 02:22:44.897522 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:44.897048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:49349)
0929 02:22:44.897253 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:44.897355 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260929 02:22:44.896533 24529 raft_consensus.cc:348] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
W20260929 02:22:44.897738 24637 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:22:44.897827 24529 raft_consensus.cc:729] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25762067c8f447c9f0a589bc17b6886, State: Initialized, Role: FOLLOWER
I20260929 02:22:44.898497 24529 consensus_queue.cc:260] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:44.899978 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:44.899630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:56273)
0929 02:22:44.899767 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:44.899835 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:22:44.900084 24529 ts_tablet_manager.cc:1428] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260929 02:22:44.900967 24639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:44.900579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:50381)
0929 02:22:44.900711 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:44.900773 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 02:22:45.238344 24641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:45.238624 24641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:45.238714 24641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:45.245885 24641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:45.246006 24641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:45.246119 24641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:22:45.255894 24641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=39015
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:45.257207 24641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:45.258527 24641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:45.271543 24646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:45.271525 24647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:45.271765 24649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:45.272488 24641 server_base.cc:1034] running on GCE node
I20260929 02:22:45.273180 24641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:45.274107 24641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:45.275305 24641 hybrid_clock.cc:648] HybridClock initialized: now 1790648565275243 us; error 52 us; skew 500 ppm
I20260929 02:22:45.275590 24641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:45 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648565, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:45.278688 24641 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:22:45.280135 24641 webserver.cc:460] Webserver started at http://127.7.170.2:39015/ using document root <none> and password file <none>
I20260929 02:22:45.280807 24641 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:45.280983 24641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:45.286665 24641 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 02:22:45.289582 24656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.291043 24641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:22:45.291205 24641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "98f15783d7e9402da21d8c1367cb6c3b"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "e2e511216eec6a59d415ff9b66d9e139"
server_key_iv: "e7ec82fc44811f4047217bc31b0f6562"
server_key_version: "encryptionkey@0"
I20260929 02:22:45.292229 24641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:45.320190 24641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:45.324291 24641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:45.324574 24641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:45.325626 24641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:45.326974 24641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:45.327073 24641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.327179 24641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:45.327247 24641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.357897 24641 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34507
I20260929 02:22:45.357970 24768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34507 every 8 connection(s)
I20260929 02:22:45.359292 24641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:22:45.359908 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24641
I20260929 02:22:45.360612 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:33877
--local_ip_for_outbound_sockets=127.7.170.3
--tserver_master_addrs=127.7.170.62:43475
--webserver_port=41579
--webserver_interface=127.7.170.3
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:35451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:22:45.364677 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.362358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:55453)
0929 02:22:45.363270 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.363546 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":368,"threads_started":1}
W20260929 02:22:45.365382 24769 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:43475 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:43475: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
I20260929 02:22:45.366290 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.365917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:40105)
0929 02:22:45.366072 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.366157 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260929 02:22:45.367130 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.366805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:42545)
0929 02:22:45.366930 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.367013 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:22:45.367357 24769 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260929 02:22:45.677601 24772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:22:45.677884 24772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:22:45.678035 24772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:22:45.685331 24772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:22:45.685436 24772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:22:45.685609 24772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:22:45.695370 24772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:35451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:33877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=41579
--tserver_master_addrs=127.7.170.62:43475
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:22:45.696704 24772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:22:45.698064 24772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:22:45.710242 24777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:45.710209 24778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:22:45.710271 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:22:45.710922 24772 server_base.cc:1034] running on GCE node
I20260929 02:22:45.711437 24772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:22:45.712267 24772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:22:45.713480 24772 hybrid_clock.cc:648] HybridClock initialized: now 1790648565713427 us; error 44 us; skew 500 ppm
I20260929 02:22:45.713773 24772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:22:45 dist-test-slave-8cs2 krb5kdc[23319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648565, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:22:45.716638 24772 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:22:45.718055 24772 webserver.cc:460] Webserver started at http://127.7.170.3:41579/ using document root <none> and password file <none>
I20260929 02:22:45.718710 24772 fs_manager.cc:359] Metadata directory not provided
I20260929 02:22:45.718833 24772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:22:45.723870 24772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260929 02:22:45.726631 24787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.727831 24772 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:22:45.727982 24772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "428363a4d3cc4ce29df73a9fe0de72fc"
format_stamp: "Formatted at 2026-09-29 02:21:59 on dist-test-slave-8cs2"
server_key: "b8bb5ec174ca504aa57c4588216973bf"
server_key_iv: "12c483cf3884e0ff071c0bfa333629ce"
server_key_version: "encryptionkey@0"
I20260929 02:22:45.728919 24772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:22:45.756121 24772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:22:45.759954 24772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:22:45.760201 24772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:22:45.761171 24772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:22:45.762437 24772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:22:45.762537 24772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.762640 24772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:22:45.762705 24772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:22:45.791368 24772 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:33877
I20260929 02:22:45.791419 24899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:33877 every 8 connection(s)
I20260929 02:22:45.792723 24772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:22:45.794821 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24772
I20260929 02:22:45.797923 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.795629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:57883)
0929 02:22:45.796531 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.796951 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":355,"threads_started":1}
W20260929 02:22:45.798658 24900 heartbeater.cc:644] Failed to heartbeat to 127.7.170.62:43475 (0 consecutive failures): Network error: Failed to ping master at 127.7.170.62:43475: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
I20260929 02:22:45.799360 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.799071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:43841)
0929 02:22:45.799197 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.799259 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:22:45.800074 24903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.799796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:38975)
0929 02:22:45.799901 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.799974 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 02:22:45.800297 24900 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260929 02:22:45.902400 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:45.901692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:39803)
0929 02:22:45.902133 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:45.902214 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260929 02:22:46.048070 24905 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:46.048367 24905 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:46.050217 24905 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
Sep 29 02:22:46 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648564, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
Sep 29 02:22:46 dist-test-slave-8cs2 krb5kdc[23319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648564, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:22:46.077199 24904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.050627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:33877 (local address 127.7.170.1:42217)
0929 02:22:46.050943 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:46.050984 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:46.051585 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:46.055288 (+ 3703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:46.055298 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:46.056066 (+ 768us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:46.056766 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:46.056790 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.058321 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.058327 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:46.059720 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:46.059731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.060770 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.060777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:46.061212 (+ 435us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:46.066457 (+ 5245us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:46.066504 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:46.069355 (+ 2851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:46.073526 (+ 4171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:46.073533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:46.073544 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:46.073988 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:46.074421 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074425 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:46.074428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:46.074617 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:46.075559 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:46.075564 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:46.075855 (+ 291us) client_negotiation.cc:769] Sending connection context
0929 02:22:46.076157 (+ 302us) client_negotiation.cc:240] Negotiation successful
0929 02:22:46.076798 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":313,"spinlock_wait_cycles":5632}
I20260929 02:22:46.077288 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.050887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:42217 (local address 127.7.170.3:33877)
0929 02:22:46.052778 (+ 1891us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:46.052790 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:46.052831 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:22:46.054452 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:46.054470 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:46.054829 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:46.055194 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:46.056985 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.058139 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.059847 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.060586 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.060753 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:46.069722 (+ 8969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:46.069736 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:46.069758 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:46.069809 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:46.073392 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074104 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:46.074112 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:46.074118 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:46.074277 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074716 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:46.074721 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:46.074724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:46.075430 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:46.075600 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:46.076450 (+ 850us) server_negotiation.cc:299] Negotiation successful
0929 02:22:46.076865 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1714,"thread_start_us":1555,"threads_started":1}
I20260929 02:22:46.077288 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.050807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:39733 (local address 127.7.170.2:34507)
0929 02:22:46.052273 (+ 1466us) server_negotiation.cc:206] Beginning negotiation
0929 02:22:46.052289 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:22:46.052773 (+ 484us) server_negotiation.cc:407] Connection header received
0929 02:22:46.054415 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:22:46.054424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:22:46.054832 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:22:46.055114 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:22:46.056982 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.058189 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.059853 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.060606 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.060766 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:46.069922 (+ 9156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:22:46.069931 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:22:46.069942 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:22:46.069986 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:22:46.073393 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074562 (+ 1169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:46.074569 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:46.074574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:46.074776 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:22:46.075111 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:22:46.075115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:22:46.075119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:22:46.075728 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:22:46.075876 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0929 02:22:46.076446 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 02:22:46.076852 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1308,"thread_start_us":1159,"threads_started":1}
I20260929 02:22:46.077214 24906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.050654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34507 (local address 127.7.170.1:39733)
0929 02:22:46.052654 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:46.052671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0929 02:22:46.052791 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:22:46.055575 (+ 2784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:22:46.055578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:22:46.056066 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:22:46.056766 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:46.056791 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.058342 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.058346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:46.059723 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:22:46.059735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:22:46.061062 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:22:46.061068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:22:46.061213 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:22:46.066457 (+ 5244us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:22:46.066505 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:22:46.069664 (+ 3159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:22:46.074099 (+ 4435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:46.074104 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:46.074431 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:46.074884 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:22:46.074894 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:22:46.074897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:22:46.075018 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:22:46.075850 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:22:46.075854 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:22:46.076020 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 02:22:46.076157 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 02:22:46.076680 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1886,"mutex_wait_us":317,"spinlock_wait_cycles":100864,"thread_start_us":1665,"threads_started":1}
W20260929 02:22:46.080327 24524 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
W20260929 02:22:46.080561 24525 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
I20260929 02:22:46.080698 24525 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f25762067c8f447c9f0a589bc17b6886; no voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b
I20260929 02:22:46.081099 24905 raft_consensus.cc:2726] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:22:46.368824 24909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.367925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:45883)
0929 02:22:46.368423 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:46.368563 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":251,"threads_started":1}
I20260929 02:22:46.801584 24910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.800790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:59873)
0929 02:22:46.801263 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:46.801392 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":267,"threads_started":1}
I20260929 02:22:46.903950 24911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:46.903222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:40341)
0929 02:22:46.903675 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:46.903760 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":244,"threads_started":1}
I20260929 02:22:47.370522 24912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:47.369770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:54427)
0929 02:22:47.370205 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:47.370291 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":232,"threads_started":1}
I20260929 02:22:47.803385 24913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:47.802542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:53861)
0929 02:22:47.803052 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:47.803168 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":264,"threads_started":1}
I20260929 02:22:47.905750 24914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:47.904830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:42337)
0929 02:22:47.905420 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:47.905513 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":296,"threads_started":1}
I20260929 02:22:48.094606 24915 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:48.094889 24915 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:48.095767 24915 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
W20260929 02:22:48.096920 24524 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
W20260929 02:22:48.097169 24525 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
I20260929 02:22:48.097265 24525 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f25762067c8f447c9f0a589bc17b6886; no voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b
I20260929 02:22:48.097532 24915 raft_consensus.cc:2726] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:22:48.372594 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:48.371723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:38779)
0929 02:22:48.372277 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:48.372396 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":289,"threads_started":1}
I20260929 02:22:48.805442 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:48.804493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:38227)
0929 02:22:48.805086 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:48.805237 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":296,"threads_started":1}
I20260929 02:22:48.907799 24918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:48.906902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:56871)
0929 02:22:48.907482 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:48.907592 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":311,"threads_started":1}
I20260929 02:22:49.374557 24919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:49.373762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:36809)
0929 02:22:49.374240 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:49.374336 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":233,"threads_started":1}
I20260929 02:22:49.807267 24920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:49.806455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:33981)
0929 02:22:49.806941 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:49.807055 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":233,"threads_started":1}
I20260929 02:22:49.909752 24921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:49.908953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:49145)
0929 02:22:49.909455 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:49.909558 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":243,"threads_started":1}
I20260929 02:22:50.376304 24922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:50.375573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:33189)
0929 02:22:50.376042 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:50.376132 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":235,"threads_started":1}
I20260929 02:22:50.809089 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:50.808330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:51849)
0929 02:22:50.808790 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:50.808887 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":257,"threads_started":1}
I20260929 02:22:50.911257 24924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:50.910554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:47143)
0929 02:22:50.911019 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:50.911097 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":237,"threads_started":1}
I20260929 02:22:51.377907 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:51.377187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:47489)
0929 02:22:51.377624 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:51.377702 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260929 02:22:51.462611 24926 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:51.462821 24926 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:51.463547 24926 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
W20260929 02:22:51.464731 24525 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
W20260929 02:22:51.465021 24524 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
I20260929 02:22:51.465148 24524 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f25762067c8f447c9f0a589bc17b6886; no voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b
I20260929 02:22:51.465401 24926 raft_consensus.cc:2726] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:22:51.810819 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:51.809954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:51799)
0929 02:22:51.810513 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:51.810606 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":309,"threads_started":1}
I20260929 02:22:51.913071 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:51.912196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:46463)
0929 02:22:51.912700 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:51.912797 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":252,"threads_started":1}
I20260929 02:22:52.379846 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:52.378908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:43867)
0929 02:22:52.379510 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:52.379605 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":275,"threads_started":1}
I20260929 02:22:52.812726 24930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:52.811837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:42635)
0929 02:22:52.812406 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:52.812502 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":242,"threads_started":1}
I20260929 02:22:52.914855 24931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:52.914097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:54217)
0929 02:22:52.914552 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:52.914654 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260929 02:22:53.381769 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:53.380864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:38239)
0929 02:22:53.381457 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:53.381560 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":282,"threads_started":1}
I20260929 02:22:53.814852 24933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:53.813907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:36355)
0929 02:22:53.814488 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:53.814597 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":280,"threads_started":1}
I20260929 02:22:53.886159 24934 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:53.886394 24934 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:53.887261 24934 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
W20260929 02:22:53.888310 24524 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
W20260929 02:22:53.888478 24525 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
I20260929 02:22:53.888607 24525 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f25762067c8f447c9f0a589bc17b6886; no voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b
I20260929 02:22:53.888856 24934 raft_consensus.cc:2726] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:22:53.916853 24935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:53.915798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:58159)
0929 02:22:53.916424 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:53.916530 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":314,"threads_started":1}
I20260929 02:22:54.383812 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:54.382864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:38257)
0929 02:22:54.383488 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:54.383602 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":357,"threads_started":1}
I20260929 02:22:54.816848 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:54.815936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:53281)
0929 02:22:54.816527 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:54.816649 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":287,"threads_started":1}
I20260929 02:22:54.918855 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:54.917995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:45313)
0929 02:22:54.918524 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:54.918636 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":275,"threads_started":1}
I20260929 02:22:55.385979 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:55.385069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:41489)
0929 02:22:55.385622 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:55.385729 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":278,"threads_started":1}
I20260929 02:22:55.818883 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:55.817994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:54087)
0929 02:22:55.818557 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:55.818645 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":329,"threads_started":1}
I20260929 02:22:55.920758 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:55.919916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:39795)
0929 02:22:55.920424 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:55.920517 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":259,"threads_started":1}
I20260929 02:22:56.388154 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:56.387222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:54965)
0929 02:22:56.387822 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:56.387929 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":299,"threads_started":1}
I20260929 02:22:56.820736 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:56.819869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:39813)
0929 02:22:56.820414 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:56.820509 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":271,"threads_started":1}
I20260929 02:22:56.922663 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:56.921761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:41527)
0929 02:22:56.922359 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:56.922451 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":334,"threads_started":1}
I20260929 02:22:57.390173 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:57.389306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:41335)
0929 02:22:57.389859 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:57.389948 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":305,"threads_started":1}
I20260929 02:22:57.822749 24946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:57.821860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:47663)
0929 02:22:57.822424 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:57.822522 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":292,"threads_started":1}
I20260929 02:22:57.924623 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:57.923729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:58423)
0929 02:22:57.924277 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:57.924376 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":271,"threads_started":1}
I20260929 02:22:58.379808 24948 raft_consensus.cc:482] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:22:58.380126 24948 raft_consensus.cc:504] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "98f15783d7e9402da21d8c1367cb6c3b" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34507 } } peers { permanent_uuid: "428363a4d3cc4ce29df73a9fe0de72fc" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 33877 } } peers { permanent_uuid: "f25762067c8f447c9f0a589bc17b6886" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46555 } }
I20260929 02:22:58.381083 24948 leader_election.cc:288] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507), 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877)
W20260929 02:22:58.382200 24525 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 428363a4d3cc4ce29df73a9fe0de72fc (127.7.170.3:33877): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
W20260929 02:22:58.382373 24524 leader_election.cc:341] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 98f15783d7e9402da21d8c1367cb6c3b (127.7.170.2:34507): Not found: Tablet not found: cd11c382a01244049d32b4d244e32911
I20260929 02:22:58.382493 24524 leader_election.cc:302] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f25762067c8f447c9f0a589bc17b6886; no voters: 428363a4d3cc4ce29df73a9fe0de72fc, 98f15783d7e9402da21d8c1367cb6c3b
I20260929 02:22:58.382768 24948 raft_consensus.cc:2726] T cd11c382a01244049d32b4d244e32911 P f25762067c8f447c9f0a589bc17b6886 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260929 02:22:58.392042 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:58.391170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:49257)
0929 02:22:58.391720 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:58.391835 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":271,"threads_started":1}
I20260929 02:22:58.824712 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:58.823809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:60681)
0929 02:22:58.824387 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:58.824494 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":319,"threads_started":1}
I20260929 02:22:58.926862 24951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:58.925752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:60637)
0929 02:22:58.926491 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:58.926606 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":480,"threads_started":1}
I20260929 02:22:59.394095 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:59.393178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:52865)
0929 02:22:59.393748 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:59.393850 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":289,"threads_started":1}
I20260929 02:22:59.826694 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:59.825757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:49391)
0929 02:22:59.826356 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:59.826453 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":301,"threads_started":1}
I20260929 02:22:59.929102 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:22:59.928174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.1:40187)
0929 02:22:59.928735 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0929 02:22:59.928835 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":276,"threads_started":1}
I20260929 02:23:00.396344 24955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:00.395455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.2:40307)
0929 02:23:00.395948 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:00.396131 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":249,"threads_started":1}
I20260929 02:23:00.799237 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24467
I20260929 02:23:00.815312 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:00.800620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.7.170.1:46555)
0929 02:23:00.801206 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:00.801215 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:00.801246 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:23:00.801321 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:00.801329 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:00.801601 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:00.801756 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:00.803314 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:00.804397 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:00.805806 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:00.806487 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:00.806588 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:00.809362 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:00.809374 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:00.809393 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:00.809434 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:00.812421 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:00.813149 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:00.813157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:00.813160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:00.813312 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:00.813704 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:00.813707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:00.813709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:00.814087 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:00.814347 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:00.814647 (+ 300us) server_negotiation.cc:299] Negotiation successful
0929 02:23:00.814912 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":270,"threads_started":1}
I20260929 02:23:00.816747 24632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:00.829051 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:00.828064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:43475 (local address 127.7.170.3:50625)
0929 02:23:00.828687 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:00.828812 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.62:43475: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":323,"threads_started":1}
I20260929 02:23:00.912591 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24467
I20260929 02:23:00.931413 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24641
I20260929 02:23:00.946174 24959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:00.932293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.7.170.2:34507)
0929 02:23:00.932892 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:00.932897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:00.932916 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:23:00.933011 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:00.933015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:00.933086 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:00.933199 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:00.934554 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:00.935502 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:00.937357 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:00.937836 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:00.937944 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:00.940870 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:00.940875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:00.940879 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:00.940918 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:00.943655 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:00.944323 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:00.944327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:00.944329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:00.944453 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:00.944824 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:00.944828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:00.944830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:00.945149 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:00.945367 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:00.945659 (+ 292us) server_negotiation.cc:299] Negotiation successful
0929 02:23:00.945852 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":352,"threads_started":1}
I20260929 02:23:00.946964 24764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:01.034873 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24641
I20260929 02:23:01.051177 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24772
I20260929 02:23:01.065287 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:01.052181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43990 (local address 127.7.170.3:33877)
0929 02:23:01.052770 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:01.052774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:01.052793 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:23:01.052869 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:01.052872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:01.052981 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:01.053075 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:01.054535 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:01.055422 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:01.056690 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:01.057211 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:01.057314 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:01.060039 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:01.060045 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:01.060048 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:01.060089 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:01.062732 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:01.063365 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:01.063370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:01.063372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:01.063520 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:01.063901 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:01.063905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:01.063907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:01.064188 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:01.064411 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:01.064665 (+ 254us) server_negotiation.cc:299] Negotiation successful
0929 02:23:01.064864 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":300,"threads_started":1}
I20260929 02:23:01.066056 24895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:01.148293 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24772
2026-09-29T02:23:01Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64449 ms)
[----------] 30 tests from SecurityITest (284624 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:01 dist-test-slave-8cs2 krb5kdc[24965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:01 dist-test-slave-8cs2 krb5kdc[24965](info): set up 2 sockets
Sep 29 02:23:01 dist-test-slave-8cs2 krb5kdc[24965](info): commencing operation
krb5kdc: starting...
W20260929 02:23:02.189280 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.013s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:02 dist-test-slave-8cs2 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:02.454097 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:37485
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:37485
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:02.794306 24978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:02.794621 24978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:02.794710 24978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:02.802349 24978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:02.802464 24978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:02.802508 24978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:02.802568 24978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:02.802613 24978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:02.812732 24978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:37485
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:37485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:02.814078 24978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:02.815336 24978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:02.826735 24984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:02.827230 24983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:02.827425 24986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:02.827805 24978 server_base.cc:1034] running on GCE node
I20260929 02:23:02.828616 24978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:02.830113 24978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:02.831367 24978 hybrid_clock.cc:648] HybridClock initialized: now 1790648582831305 us; error 39 us; skew 500 ppm
I20260929 02:23:02.831730 24978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:02 dist-test-slave-8cs2 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:02.834805 24978 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:02.836227 24978 webserver.cc:460] Webserver started at http://127.7.170.62:42459/ using document root <none> and password file <none>
I20260929 02:23:02.836897 24978 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:02.837041 24978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:02.837325 24978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:02.840305 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b"
format_stamp: "Formatted at 2026-09-29 02:23:02 on dist-test-slave-8cs2"
server_key: "77c38d23a0d30bee32f0d660461b69d9"
server_key_iv: "cdc88c6a8c6d48e967fd6dcb6908748a"
server_key_version: "encryptionkey@0"
I20260929 02:23:02.841075 24978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b"
format_stamp: "Formatted at 2026-09-29 02:23:02 on dist-test-slave-8cs2"
server_key: "77c38d23a0d30bee32f0d660461b69d9"
server_key_iv: "cdc88c6a8c6d48e967fd6dcb6908748a"
server_key_version: "encryptionkey@0"
I20260929 02:23:02.846251 24978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:23:02.849833 24993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:02.851418 24978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:02.851613 24978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b"
format_stamp: "Formatted at 2026-09-29 02:23:02 on dist-test-slave-8cs2"
server_key: "77c38d23a0d30bee32f0d660461b69d9"
server_key_iv: "cdc88c6a8c6d48e967fd6dcb6908748a"
server_key_version: "encryptionkey@0"
I20260929 02:23:02.851809 24978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:02.876993 24978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:02.881230 24978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:02.881493 24978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:02.898845 24978 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:37485
I20260929 02:23:02.898890 25044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:37485 every 8 connection(s)
I20260929 02:23:02.900271 24978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:02.904230 25045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:02.910130 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 24978
I20260929 02:23:02.910372 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:02.911183 7848 external_mini_cluster.cc:1428] Setting key 5de9a7098af921c418dafc4a6c3143f3
I20260929 02:23:02.916741 25045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: Bootstrap starting.
I20260929 02:23:02.920771 25045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:02.922179 25045 log.cc:824] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:02.926076 25045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: No bootstrap required, opened a new log
Sep 29 02:23:02 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:02.935457 25045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } }
I20260929 02:23:02.935880 25045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:02.935985 25045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f96a8424734ad1b4ec8d54ab7dfc4b, State: Initialized, Role: FOLLOWER
I20260929 02:23:02.936561 25045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } }
I20260929 02:23:02.936762 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:02.913573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.7.170.62:37485)
0929 02:23:02.914503 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:02.914516 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:02.914569 (+ 53us) server_negotiation.cc:407] Connection header received
0929 02:23:02.916361 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:02.916389 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:02.916861 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:02.917327 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:02.919264 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:02.920702 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:02.922559 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:02.923661 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:02.923830 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:02.928989 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:02.928999 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:02.929021 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:02.929065 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:02.932264 (+ 3199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:02.933200 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:02.933207 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:02.933214 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:02.933408 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:02.933833 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:02.933837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:02.933839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:02.934437 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:02.934665 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:02.935127 (+ 462us) server_negotiation.cc:299] Negotiation successful
0929 02:23:02.935511 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":421,"threads_started":1}
I20260929 02:23:02.936813 25045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:02.936990 25045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:02.937211 25045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:02.939934 25045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } }
I20260929 02:23:02.940431 25045 leader_election.cc:302] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54f96a8424734ad1b4ec8d54ab7dfc4b; no voters:
I20260929 02:23:02.940994 25045 leader_election.cc:288] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:02.941128 25050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:02.941778 25050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [term 1 LEADER]: Becoming Leader. State: Replica: 54f96a8424734ad1b4ec8d54ab7dfc4b, State: Running, Role: LEADER
I20260929 02:23:02.942338 25050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } }
I20260929 02:23:02.942895 25045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:02.945231 25052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54f96a8424734ad1b4ec8d54ab7dfc4b. Latest consensus state: current_term: 1 leader_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } } }
I20260929 02:23:02.945639 25052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:02.946664 25051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54f96a8424734ad1b4ec8d54ab7dfc4b" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37485 } } }
I20260929 02:23:02.947042 25051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:02.948158 25058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:02.952764 25058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:02.961356 25058 catalog_manager.cc:1347] Generated new cluster ID: e8d0c1fc444047a4ab3280c7962d9d0f
I20260929 02:23:02.961483 25058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:02.985769 25058 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:02.987157 25058 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:02.992807 25058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: Generated new TSK 0
I20260929 02:23:02.993961 25058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:23:03.068775 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:03.402034 25071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:03.402323 25071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:03.402418 25071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:03.409803 25071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:03.409930 25071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:03.410045 25071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:03.420362 25071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:03.421783 25071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:03.423305 25071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:03.436219 25077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:03.436450 25079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:03.436203 25076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:03.437368 25071 server_base.cc:1034] running on GCE node
I20260929 02:23:03.438025 25071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:03.438993 25071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:03.440200 25071 hybrid_clock.cc:648] HybridClock initialized: now 1790648583440143 us; error 43 us; skew 500 ppm
I20260929 02:23:03.440508 25071 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:03 dist-test-slave-8cs2 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:03.443610 25071 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:23:03.445106 25071 webserver.cc:460] Webserver started at http://127.7.170.1:37893/ using document root <none> and password file <none>
I20260929 02:23:03.445778 25071 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:03.445899 25071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:03.446180 25071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:03.449222 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "f971f9a6481e49fa8b940930b882992f"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "d219c94717bc47320cbc25cb935169a6"
server_key_iv: "6dba8d312f41f581d76e12927aa7e5d2"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.449970 25071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "f971f9a6481e49fa8b940930b882992f"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "d219c94717bc47320cbc25cb935169a6"
server_key_iv: "6dba8d312f41f581d76e12927aa7e5d2"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.455209 25071 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:03.458766 25086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:03.460134 25071 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:23:03.460285 25071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "f971f9a6481e49fa8b940930b882992f"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "d219c94717bc47320cbc25cb935169a6"
server_key_iv: "6dba8d312f41f581d76e12927aa7e5d2"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.460481 25071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:03.481122 25071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:03.485555 25071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:03.485843 25071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:03.486795 25071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:03.488133 25071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:03.488206 25071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:03.488319 25071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:03.488370 25071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:03.517717 25071 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:36787
I20260929 02:23:03.517769 25198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:36787 every 8 connection(s)
I20260929 02:23:03.519037 25071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:03.523846 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25071
I20260929 02:23:03.524041 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:03.524668 7848 external_mini_cluster.cc:1428] Setting key f833e36d3d966d1826960fe1b97b438c
Sep 29 02:23:03 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:03.539269 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:03.521682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:53099 (local address 127.7.170.62:37485)
0929 02:23:03.522109 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:03.522113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:03.523200 (+ 1087us) server_negotiation.cc:407] Connection header received
0929 02:23:03.525227 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:03.525233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:03.525325 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:03.525418 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:03.527213 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:03.527929 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:03.529521 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:03.529918 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:03.530030 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:03.533514 (+ 3484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:03.533520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:03.533523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:03.533558 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:03.535969 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:03.536839 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:03.536843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:03.536846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:03.537025 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:03.537490 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:03.537494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:03.537496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:03.537779 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:03.537898 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:03.538818 (+ 920us) server_negotiation.cc:299] Negotiation successful
0929 02:23:03.539016 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":207,"threads_started":1}
I20260929 02:23:03.540589 25202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:03.522281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:37485 (local address 127.7.170.1:53099)
0929 02:23:03.523016 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:03.523064 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:03.524910 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:03.525707 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:03.525719 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:03.526456 (+ 737us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:03.527050 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:03.527068 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:03.528071 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:03.528077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:03.529389 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:03.529400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:03.530143 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:03.530150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:03.530381 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:03.531041 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:03.531070 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:03.533284 (+ 2214us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:03.536110 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:03.536121 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:03.536140 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:03.536702 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:03.537128 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:03.537133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:03.537137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:03.537379 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:03.537911 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:03.537920 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:03.538381 (+ 461us) client_negotiation.cc:769] Sending connection context
0929 02:23:03.538671 (+ 290us) client_negotiation.cc:240] Negotiation successful
0929 02:23:03.539228 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":285,"threads_started":1}
I20260929 02:23:03.542268 25199 heartbeater.cc:347] Connected to a master server at 127.7.170.62:37485
I20260929 02:23:03.542603 25199 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:03.543363 25199 heartbeater.cc:511] Master 127.7.170.62:37485 requested a full tablet report, sending...
I20260929 02:23:03.545548 25010 ts_manager.cc:194] Registered new tserver with Master: f971f9a6481e49fa8b940930b882992f (127.7.170.1:36787)
I20260929 02:23:03.547140 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:53099
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:23:03.584481 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:03.913429 25205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:03.913717 25205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:03.913856 25205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:03.921334 25205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:03.921480 25205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:03.921643 25205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:03.931581 25205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:03.933010 25205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:03.934337 25205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:03.947278 25211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:03.947235 25213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:03.947297 25210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:03.948181 25205 server_base.cc:1034] running on GCE node
I20260929 02:23:03.948836 25205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:03.949846 25205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:03.951056 25205 hybrid_clock.cc:648] HybridClock initialized: now 1790648583950988 us; error 58 us; skew 500 ppm
I20260929 02:23:03.951380 25205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:03 dist-test-slave-8cs2 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:03.954468 25205 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:23:03.955971 25205 webserver.cc:460] Webserver started at http://127.7.170.2:41073/ using document root <none> and password file <none>
I20260929 02:23:03.956760 25205 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:03.956868 25205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:03.957198 25205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:03.960134 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "012603199f4141708c2cb3916f7d79ab"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "1382dc77f4b6eb0336868278cbada442"
server_key_iv: "47f4c73eaf8ee723d2f580df7495996e"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.960893 25205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "012603199f4141708c2cb3916f7d79ab"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "1382dc77f4b6eb0336868278cbada442"
server_key_iv: "47f4c73eaf8ee723d2f580df7495996e"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.966116 25205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:23:03.969551 25220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:03.971024 25205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:03.971179 25205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "012603199f4141708c2cb3916f7d79ab"
format_stamp: "Formatted at 2026-09-29 02:23:03 on dist-test-slave-8cs2"
server_key: "1382dc77f4b6eb0336868278cbada442"
server_key_iv: "47f4c73eaf8ee723d2f580df7495996e"
server_key_version: "encryptionkey@0"
I20260929 02:23:03.971393 25205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:03.998046 25205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:04.002104 25205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:04.002377 25205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:04.003360 25205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:04.004742 25205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:04.004820 25205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:04.004954 25205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:04.005013 25205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260929 02:23:04.034852 25205 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:34247
I20260929 02:23:04.034889 25332 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:34247 every 8 connection(s)
I20260929 02:23:04.036092 25205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:04.039261 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25205
I20260929 02:23:04.039429 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:04.039831 7848 external_mini_cluster.cc:1428] Setting key 39a8f65dde9cc1291caca852e1878e68
Sep 29 02:23:04 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648583, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:04.057060 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.038603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56321 (local address 127.7.170.62:37485)
0929 02:23:04.038779 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.038783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.040086 (+ 1303us) server_negotiation.cc:407] Connection header received
0929 02:23:04.042489 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.042494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.042607 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.042719 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:04.044374 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.045335 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.046970 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.047554 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.047672 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.051186 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:04.051192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:04.051195 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:04.051230 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:04.053730 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.054568 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.054573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.054576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.054730 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.055202 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.055206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.055220 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.055529 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:04.055656 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.056542 (+ 886us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.056715 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:23:04.058386 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.039082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:37485 (local address 127.7.170.2:56321)
0929 02:23:04.039909 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.039961 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.042209 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.042998 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.043007 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.043510 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:04.044210 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.044229 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.045507 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.045511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.046796 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.046809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.047749 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.047754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.048038 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.048705 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:04.048728 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:04.050941 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:04.053872 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:04.053882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:04.053900 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:04.054434 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:04.054845 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:04.054850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:04.054853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:04.055074 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:04.055667 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:04.055674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:04.056137 (+ 463us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.056447 (+ 310us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.057021 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":322,"threads_started":1}
I20260929 02:23:04.059981 25333 heartbeater.cc:347] Connected to a master server at 127.7.170.62:37485
I20260929 02:23:04.060269 25333 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:04.061033 25333 heartbeater.cc:511] Master 127.7.170.62:37485 requested a full tablet report, sending...
I20260929 02:23:04.062594 25010 ts_manager.cc:194] Registered new tserver with Master: 012603199f4141708c2cb3916f7d79ab (127.7.170.2:34247)
I20260929 02:23:04.063418 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:56321
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:23:04.098305 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:04.422042 25338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:04.422410 25338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:04.422567 25338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:04.429901 25338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:04.430027 25338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:04.430192 25338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:04.440027 25338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:37485
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:04.441339 25338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:04.442711 25338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:04.455844 25343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:04.455816 25346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:04.455818 25344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:04.456537 25338 server_base.cc:1034] running on GCE node
I20260929 02:23:04.457154 25338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:04.458051 25338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:04.459254 25338 hybrid_clock.cc:648] HybridClock initialized: now 1790648584459201 us; error 39 us; skew 500 ppm
I20260929 02:23:04.459569 25338 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:04 dist-test-slave-8cs2 krb5kdc[24965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:04.462610 25338 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:23:04.464112 25338 webserver.cc:460] Webserver started at http://127.7.170.3:34927/ using document root <none> and password file <none>
I20260929 02:23:04.464774 25338 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:04.464895 25338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:04.465240 25338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:04.468111 25338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "4870c5a902af41129239c19b841fe431"
format_stamp: "Formatted at 2026-09-29 02:23:04 on dist-test-slave-8cs2"
server_key: "08c792d1f89895d4e1a54c8dfa24b30c"
server_key_iv: "eee03bc069d3ff33058d1b19fd382d20"
server_key_version: "encryptionkey@0"
I20260929 02:23:04.468894 25338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "4870c5a902af41129239c19b841fe431"
format_stamp: "Formatted at 2026-09-29 02:23:04 on dist-test-slave-8cs2"
server_key: "08c792d1f89895d4e1a54c8dfa24b30c"
server_key_iv: "eee03bc069d3ff33058d1b19fd382d20"
server_key_version: "encryptionkey@0"
I20260929 02:23:04.473963 25338 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:23:04.477421 25353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:04.478772 25338 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:23:04.478935 25338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "4870c5a902af41129239c19b841fe431"
format_stamp: "Formatted at 2026-09-29 02:23:04 on dist-test-slave-8cs2"
server_key: "08c792d1f89895d4e1a54c8dfa24b30c"
server_key_iv: "eee03bc069d3ff33058d1b19fd382d20"
server_key_version: "encryptionkey@0"
I20260929 02:23:04.479127 25338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:04.498196 25338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:04.502676 25338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:04.502947 25338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:04.503926 25338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:04.505373 25338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:04.505450 25338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:04.505577 25338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:04.505631 25338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:04.537370 25338 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:45887
I20260929 02:23:04.537421 25465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:45887 every 8 connection(s)
I20260929 02:23:04.538679 25338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:04.542625 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25338
I20260929 02:23:04.542809 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:04.543365 7848 external_mini_cluster.cc:1428] Setting key 22edb8fbd2b2bffecb8f66a7d00e9926
I20260929 02:23:04.550334 25199 heartbeater.cc:503] Master 127.7.170.62:37485 was elected leader, sending a full tablet report...
Sep 29 02:23:04 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648584, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:04.559607 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.541193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:45625 (local address 127.7.170.62:37485)
0929 02:23:04.541361 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.541365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.542744 (+ 1379us) server_negotiation.cc:407] Connection header received
0929 02:23:04.545047 (+ 2303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.545051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.545148 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.545244 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:04.547078 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.547888 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.549507 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.549913 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.550023 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.553829 (+ 3806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:04.553834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:04.553843 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:04.553877 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:04.556411 (+ 2534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.557190 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.557195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.557197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.557326 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.557800 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.557803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.557805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.558085 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:04.558215 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.559041 (+ 826us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.559259 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:23:04.560835 25468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.541710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:37485 (local address 127.7.170.3:45625)
0929 02:23:04.542531 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.542592 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.544726 (+ 2134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.545524 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.545537 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.546311 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:04.546914 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.546933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.548030 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.548034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.549370 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.549384 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.550134 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.550142 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.550380 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.551070 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:04.551109 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:04.553621 (+ 2512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:04.556549 (+ 2928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:04.556558 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:04.556576 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:04.557069 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:04.557433 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:04.557438 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:04.557442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:04.557690 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:04.558207 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:04.558215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:04.558655 (+ 440us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.558925 (+ 270us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.559489 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":286,"threads_started":1}
I20260929 02:23:04.562366 25466 heartbeater.cc:347] Connected to a master server at 127.7.170.62:37485
I20260929 02:23:04.562655 25466 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:04.563396 25466 heartbeater.cc:511] Master 127.7.170.62:37485 requested a full tablet report, sending...
I20260929 02:23:04.564824 25010 ts_manager.cc:194] Registered new tserver with Master: 4870c5a902af41129239c19b841fe431 (127.7.170.3:45887)
I20260929 02:23:04.565713 25010 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:45625
I20260929 02:23:04.571220 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:04.588212 25201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.575680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.7.170.62:37485)
0929 02:23:04.575830 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.575836 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.576214 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:23:04.576432 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.576437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.576515 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.576602 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:04.578069 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.578783 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.580116 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.580526 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.580628 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.583095 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:04.583100 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:04.583103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:04.583142 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:04.585772 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.586410 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.586414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.586416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.586547 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:04.586907 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:04.586911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:04.586913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:04.587227 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:04.587420 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.587769 (+ 349us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.587969 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:23:04.592211 25010 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:04.599140 25010 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:04.624624 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.612848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.7.170.2:34247)
0929 02:23:04.613574 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.613583 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.614344 (+ 761us) server_negotiation.cc:407] Connection header received
0929 02:23:04.614647 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.614657 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.614966 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.615146 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:04.616140 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.617957 (+ 1817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.621051 (+ 3094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.622888 (+ 1837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.623024 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.623255 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.624028 (+ 773us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.624241 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":425,"threads_started":1}
I20260929 02:23:04.625486 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.612073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:34247 (local address 127.0.0.1:36790)
0929 02:23:04.614213 (+ 2140us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.614238 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.614454 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.615501 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.615510 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.615545 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:04.615941 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.615955 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.618187 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.618193 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.620778 (+ 2585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.620797 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.623459 (+ 2662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.623467 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.623720 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.623774 (+ 54us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.624855 (+ 1081us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.625127 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2001,"thread_start_us":329,"threads_started":1}
I20260929 02:23:04.629644 25268 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 (DEFAULT_TABLE table=test-table [id=efceb338569c44e584c2fcb119c94172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:04.631148 25268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7039962ff64f7f8f00755754bf3f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:04.632668 25479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.611946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:36787 (local address 127.0.0.1:46762)
0929 02:23:04.612763 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.612811 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.613014 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.625497 (+ 12483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.625504 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.625539 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:04.626001 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.626012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.627960 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.627966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.630080 (+ 2114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.630093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.631866 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.631873 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.632035 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.632068 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.632165 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.632307 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":2170,"threads_started":1}
I20260929 02:23:04.633041 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.612430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.7.170.1:36787)
0929 02:23:04.624394 (+ 11964us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.624405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.624443 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:23:04.624556 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.624565 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.625219 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.626102 (+ 883us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:04.626335 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.627779 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.630241 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.631635 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.631806 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.632025 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.632225 (+ 200us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.632697 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11723,"thread_start_us":12408,"threads_started":1}
I20260929 02:23:04.634919 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.613680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:45887 (local address 127.0.0.1:44970)
0929 02:23:04.625743 (+ 12063us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.625761 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.625889 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.627487 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.627491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.627515 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:04.627726 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.627736 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.629834 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.629839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.631600 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.631613 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.634158 (+ 2545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.634164 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.634361 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.634395 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.634479 (+ 84us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.634618 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11949,"thread_start_us":13170,"threads_started":1}
I20260929 02:23:04.634982 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.613982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44970 (local address 127.7.170.3:45887)
0929 02:23:04.626912 (+ 12930us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.626920 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.626953 (+ 33us) server_negotiation.cc:407] Connection header received
0929 02:23:04.627052 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.627060 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.627282 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.627483 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:04.627964 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.629650 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.631747 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.633701 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.633841 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.634066 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.634546 (+ 480us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.634695 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12767,"thread_start_us":12633,"threads_started":1}
I20260929 02:23:04.636793 25134 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 (DEFAULT_TABLE table=test-table [id=efceb338569c44e584c2fcb119c94172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:04.638433 25134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7039962ff64f7f8f00755754bf3f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:04.638509 25401 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 (DEFAULT_TABLE table=test-table [id=efceb338569c44e584c2fcb119c94172]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:04.639688 25401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7039962ff64f7f8f00755754bf3f47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:04.647222 25484 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Bootstrap starting.
I20260929 02:23:04.651450 25484 tablet_bootstrap.cc:654] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:04.652418 25486 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Bootstrap starting.
I20260929 02:23:04.653110 25484 log.cc:824] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:04.654402 25485 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Bootstrap starting.
I20260929 02:23:04.656229 25486 tablet_bootstrap.cc:654] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:04.656592 25484 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: No bootstrap required, opened a new log
I20260929 02:23:04.656860 25484 ts_tablet_manager.cc:1397] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260929 02:23:04.657415 25485 tablet_bootstrap.cc:654] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:04.658329 25486 log.cc:824] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:04.658412 25485 log.cc:824] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:04.661384 25485 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: No bootstrap required, opened a new log
I20260929 02:23:04.661383 25486 tablet_bootstrap.cc:492] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: No bootstrap required, opened a new log
I20260929 02:23:04.661661 25486 ts_tablet_manager.cc:1397] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260929 02:23:04.661659 25485 ts_tablet_manager.cc:1397] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260929 02:23:04.664790 25484 raft_consensus.cc:348] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.665184 25484 raft_consensus.cc:374] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:04.665273 25484 raft_consensus.cc:729] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 012603199f4141708c2cb3916f7d79ab, State: Initialized, Role: FOLLOWER
I20260929 02:23:04.665872 25484 consensus_queue.cc:260] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.669734 25333 heartbeater.cc:503] Master 127.7.170.62:37485 was elected leader, sending a full tablet report...
I20260929 02:23:04.670481 25484 ts_tablet_manager.cc:1428] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:23:04.672288 25486 raft_consensus.cc:348] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.672833 25486 raft_consensus.cc:374] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:04.672655 25485 raft_consensus.cc:348] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.672989 25486 raft_consensus.cc:729] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4870c5a902af41129239c19b841fe431, State: Initialized, Role: FOLLOWER
I20260929 02:23:04.673112 25485 raft_consensus.cc:374] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:04.673207 25485 raft_consensus.cc:729] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f971f9a6481e49fa8b940930b882992f, State: Initialized, Role: FOLLOWER
I20260929 02:23:04.673692 25485 consensus_queue.cc:260] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.673657 25486 consensus_queue.cc:260] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.674830 25466 heartbeater.cc:503] Master 127.7.170.62:37485 was elected leader, sending a full tablet report...
I20260929 02:23:04.675529 25486 ts_tablet_manager.cc:1428] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260929 02:23:04.678087 25485 ts_tablet_manager.cc:1428] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Time spent starting tablet: real 0.016s user 0.009s sys 0.004s
W20260929 02:23:04.771416 25200 tablet.cc:2369] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:23:04.788025 25334 tablet.cc:2369] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260929 02:23:04.790227 25467 tablet.cc:2369] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:04.981426 25490 raft_consensus.cc:482] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:04.981751 25490 raft_consensus.cc:504] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.983919 25490 leader_election.cc:288] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f971f9a6481e49fa8b940930b882992f (127.7.170.1:36787), 4870c5a902af41129239c19b841fe431 (127.7.170.3:45887)
I20260929 02:23:04.992606 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.984574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:57289 (local address 127.7.170.3:45887)
0929 02:23:04.984760 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.984766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.985595 (+ 829us) server_negotiation.cc:407] Connection header received
0929 02:23:04.985812 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.985818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.985966 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.986112 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:04.986658 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.987605 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.990028 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.991273 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.991412 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.991517 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.992219 (+ 702us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.992365 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:23:04.992638 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.984962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:35515 (local address 127.7.170.1:36787)
0929 02:23:04.985163 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:04.985170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:04.985188 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:23:04.985281 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:04.985286 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:04.985362 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:04.985447 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:04.986247 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.987304 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.990020 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.991270 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.991406 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.991517 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:04.992253 (+ 736us) server_negotiation.cc:299] Negotiation successful
0929 02:23:04.992407 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:23:04.992731 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.984188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:36787 (local address 127.7.170.2:35515)
0929 02:23:04.984907 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.984952 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.985103 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.985621 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.985626 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.985654 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:04.986075 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.986088 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.987432 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.987437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.989825 (+ 2388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.989842 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.991720 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.991727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.991983 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.992061 (+ 78us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.992197 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.992343 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":73,"thread_start_us":384,"threads_started":1}
I20260929 02:23:04.993300 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:04.984618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:45887 (local address 127.7.170.2:57289)
0929 02:23:04.985493 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:04.985507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:04.985639 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:04.986122 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:04.986126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:04.986150 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:04.986506 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.986518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.987724 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.987729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.989826 (+ 2097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:04.989851 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:04.991725 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:04.991730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:04.991987 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:04.992065 (+ 78us) client_negotiation.cc:769] Sending connection context
0929 02:23:04.992811 (+ 746us) client_negotiation.cc:240] Negotiation successful
0929 02:23:04.992969 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":76,"thread_start_us":554,"threads_started":1}
I20260929 02:23:04.993767 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7039962ff64f7f8f00755754bf3f47" candidate_uuid: "012603199f4141708c2cb3916f7d79ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f971f9a6481e49fa8b940930b882992f" is_pre_election: true
I20260929 02:23:04.994169 25421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7039962ff64f7f8f00755754bf3f47" candidate_uuid: "012603199f4141708c2cb3916f7d79ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4870c5a902af41129239c19b841fe431" is_pre_election: true
I20260929 02:23:04.994273 25154 raft_consensus.cc:2454] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012603199f4141708c2cb3916f7d79ab in term 0.
I20260929 02:23:04.994591 25421 raft_consensus.cc:2454] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 012603199f4141708c2cb3916f7d79ab in term 0.
I20260929 02:23:04.994966 25222 leader_election.cc:302] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012603199f4141708c2cb3916f7d79ab, f971f9a6481e49fa8b940930b882992f; no voters:
I20260929 02:23:04.995362 25490 raft_consensus.cc:2781] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:04.995483 25490 raft_consensus.cc:482] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:04.995581 25490 raft_consensus.cc:3037] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:04.997949 25490 raft_consensus.cc:504] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:04.998558 25490 leader_election.cc:288] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [CANDIDATE]: Term 1 election: Requested vote from peers f971f9a6481e49fa8b940930b882992f (127.7.170.1:36787), 4870c5a902af41129239c19b841fe431 (127.7.170.3:45887)
I20260929 02:23:04.999040 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7039962ff64f7f8f00755754bf3f47" candidate_uuid: "012603199f4141708c2cb3916f7d79ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f971f9a6481e49fa8b940930b882992f"
I20260929 02:23:04.999092 25421 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7039962ff64f7f8f00755754bf3f47" candidate_uuid: "012603199f4141708c2cb3916f7d79ab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4870c5a902af41129239c19b841fe431"
I20260929 02:23:04.999236 25154 raft_consensus.cc:3037] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:04.999272 25421 raft_consensus.cc:3037] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:05.001564 25421 raft_consensus.cc:2454] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012603199f4141708c2cb3916f7d79ab in term 1.
I20260929 02:23:05.001569 25154 raft_consensus.cc:2454] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 012603199f4141708c2cb3916f7d79ab in term 1.
I20260929 02:23:05.002012 25223 leader_election.cc:302] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 012603199f4141708c2cb3916f7d79ab, 4870c5a902af41129239c19b841fe431; no voters:
I20260929 02:23:05.002379 25490 raft_consensus.cc:2781] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:05.002871 25490 raft_consensus.cc:686] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 1 LEADER]: Becoming Leader. State: Replica: 012603199f4141708c2cb3916f7d79ab, State: Running, Role: LEADER
I20260929 02:23:05.003383 25490 consensus_queue.cc:237] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } }
I20260929 02:23:05.008392 25009 catalog_manager.cc:5495] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab reported cstate change: term changed from 0 to 1, leader changed from <none> to 012603199f4141708c2cb3916f7d79ab (127.7.170.2). New cstate: current_term: 1 leader_uuid: "012603199f4141708c2cb3916f7d79ab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "012603199f4141708c2cb3916f7d79ab" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 34247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 } health_report { overall_health: UNKNOWN } } }
I20260929 02:23:05.095379 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:05.089188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36800 (local address 127.7.170.2:34247)
0929 02:23:05.089407 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:05.089412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:05.089605 (+ 193us) server_negotiation.cc:407] Connection header received
0929 02:23:05.089872 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:05.089877 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:05.089978 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:05.090160 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:05.090581 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.091373 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.093044 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.093503 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.093602 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:05.094069 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:05.094759 (+ 690us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:05.094892 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:05.095022 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 02:23:05.095110 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:23:05.105333 25421 raft_consensus.cc:1261] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 1 FOLLOWER]: Refusing update from remote peer 012603199f4141708c2cb3916f7d79ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:05.105333 25154 raft_consensus.cc:1261] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 1 FOLLOWER]: Refusing update from remote peer 012603199f4141708c2cb3916f7d79ab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:05.106379 25490 consensus_queue.cc:1035] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "f971f9a6481e49fa8b940930b882992f" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:05.106683 25495 consensus_queue.cc:1035] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [LEADER]: Connected to new peer: Peer: permanent_uuid: "4870c5a902af41129239c19b841fe431" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 45887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:05.119032 25499 mvcc.cc:205] Tried to move back new op lower bound from 7334496604580106240 to 7334496604178391040. Current Snapshot: MvccSnapshot[applied={T|T < 7334496604580106240}]
I20260929 02:23:05.136162 25009 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:05.136488 25009 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:05.140379 25009 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54f96a8424734ad1b4ec8d54ab7dfc4b: Sending DeleteTablet for 3 replicas of tablet 0f7039962ff64f7f8f00755754bf3f47
I20260929 02:23:05.141620 25268 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:05 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:36790
I20260929 02:23:05.141649 25134 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:05 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:46762
I20260929 02:23:05.142436 25401 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7039962ff64f7f8f00755754bf3f47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:05 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:44970
I20260929 02:23:05.142728 25509 tablet_replica.cc:330] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: stopping tablet replica
I20260929 02:23:05.143317 25509 raft_consensus.cc:2229] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:05.144387 25509 raft_consensus.cc:2258] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:05.144686 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25071
I20260929 02:23:05.154454 25511 tablet_replica.cc:330] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: stopping tablet replica
I20260929 02:23:05.154930 25509 ts_tablet_manager.cc:1905] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:05.154985 25511 raft_consensus.cc:2229] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:05.155655 25511 raft_consensus.cc:2258] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:05.158223 25511 ts_tablet_manager.cc:1905] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:05.163136 25509 ts_tablet_manager.cc:1918] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:05.163347 25509 log.cc:1196] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/0f7039962ff64f7f8f00755754bf3f47
I20260929 02:23:05.163960 25509 ts_tablet_manager.cc:1939] T 0f7039962ff64f7f8f00755754bf3f47 P 012603199f4141708c2cb3916f7d79ab: Deleting consensus metadata
I20260929 02:23:05.164827 25510 tablet_replica.cc:330] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: stopping tablet replica
I20260929 02:23:05.165398 25510 raft_consensus.cc:2229] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:05.166327 24994 catalog_manager.cc:4841] TS 012603199f4141708c2cb3916f7d79ab (127.7.170.2:34247): tablet 0f7039962ff64f7f8f00755754bf3f47 (table test-table [id=efceb338569c44e584c2fcb119c94172]) successfully deleted
I20260929 02:23:05.166654 25510 raft_consensus.cc:2258] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:05.167966 25511 ts_tablet_manager.cc:1918] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:05.168130 25511 log.cc:1196] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/0f7039962ff64f7f8f00755754bf3f47
I20260929 02:23:05.168563 25511 ts_tablet_manager.cc:1939] T 0f7039962ff64f7f8f00755754bf3f47 P 4870c5a902af41129239c19b841fe431: Deleting consensus metadata
I20260929 02:23:05.169313 25510 ts_tablet_manager.cc:1905] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:05.170490 24996 catalog_manager.cc:4841] TS 4870c5a902af41129239c19b841fe431 (127.7.170.3:45887): tablet 0f7039962ff64f7f8f00755754bf3f47 (table test-table [id=efceb338569c44e584c2fcb119c94172]) successfully deleted
Sep 29 02:23:05 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:23:05.176811 25510 ts_tablet_manager.cc:1918] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:05.177008 25510 log.cc:1196] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/0f7039962ff64f7f8f00755754bf3f47
I20260929 02:23:05.177563 25510 ts_tablet_manager.cc:1939] T 0f7039962ff64f7f8f00755754bf3f47 P f971f9a6481e49fa8b940930b882992f: Deleting consensus metadata
I20260929 02:23:05.179522 24995 catalog_manager.cc:4841] TS f971f9a6481e49fa8b940930b882992f (127.7.170.1:36787): tablet 0f7039962ff64f7f8f00755754bf3f47 (table test-table [id=efceb338569c44e584c2fcb119c94172]) successfully deleted
I20260929 02:23:05.179833 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:05.146169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46764 (local address 127.7.170.1:36787)
0929 02:23:05.146393 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:05.146400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:05.153527 (+ 7127us) server_negotiation.cc:407] Connection header received
0929 02:23:05.153781 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:05.153788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:05.153889 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:05.153992 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:05.162212 (+ 8220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.163638 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.165831 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.166473 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.166576 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:05.173237 (+ 6661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:05.173250 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:05.173267 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:05.173307 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:05.176620 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.177401 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.177409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.177412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.177562 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.177953 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.177956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.177958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.178344 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:05.178707 (+ 363us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:05.179172 (+ 465us) server_negotiation.cc:299] Negotiation successful
0929 02:23:05.179402 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:23:05.180433 25194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:05.276142 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25071
I20260929 02:23:05.293879 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25205
Sep 29 02:23:05 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:23:05.312630 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:05.294776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.7.170.2:34247)
0929 02:23:05.294991 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:05.294996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:05.295102 (+ 106us) server_negotiation.cc:407] Connection header received
0929 02:23:05.295297 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:05.295302 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:05.295393 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:05.295485 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:05.297383 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.298279 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.300264 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.300728 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.300827 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:05.306338 (+ 5511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:05.306358 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:05.306379 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:05.306417 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:05.309766 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.310449 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.310458 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.310461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.310598 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.310955 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.310959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.310961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.311399 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:05.311753 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:05.312009 (+ 256us) server_negotiation.cc:299] Negotiation successful
0929 02:23:05.312251 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260929 02:23:05.313503 25328 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:05.438323 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25205
I20260929 02:23:05.455979 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25338
Sep 29 02:23:05 dist-test-slave-8cs2 krb5kdc[24965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:23:05.471947 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:05.456849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44976 (local address 127.7.170.3:45887)
0929 02:23:05.457066 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:05.457070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:05.457110 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:23:05.457298 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:05.457302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:05.457376 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:05.457462 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:05.458908 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.459729 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.461179 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:05.461588 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:05.461698 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:05.466316 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:05.466334 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:05.466356 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:05.466405 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:05.469339 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.470005 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.470014 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.470016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.470171 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:05.470470 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:05.470473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:05.470475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:05.470883 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:05.471115 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:05.471388 (+ 273us) server_negotiation.cc:299] Negotiation successful
0929 02:23:05.471561 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260929 02:23:05.472661 25461 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:05.565002 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25338
I20260929 02:23:05.581458 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 24978
I20260929 02:23:05.582979 25040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:05.705145 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 24978
2026-09-29T02:23:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4565 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-29T02:23:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:05Z Disabled control of system clock
I20260929 02:23:05.792745 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:35801
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:35801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:06.159449 25520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:06.159757 25520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:06.159837 25520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:06.166999 25520 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:06.167106 25520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:06.167143 25520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:06.167199 25520 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:06.167241 25520 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:06.176383 25520 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:35801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:35801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:06.177775 25520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:06.179761 25520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:06.192677 25526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:06.193585 25520 server_base.cc:1034] running on GCE node
W20260929 02:23:06.193789 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:06.193898 25525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:06.194775 25520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:06.196252 25520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:06.197458 25520 hybrid_clock.cc:648] HybridClock initialized: now 1790648586197381 us; error 59 us; skew 500 ppm
I20260929 02:23:06.197865 25520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:06.200798 25520 webserver.cc:460] Webserver started at http://127.7.170.62:37689/ using document root <none> and password file <none>
I20260929 02:23:06.201663 25520 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:06.201766 25520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:06.202099 25520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:06.205871 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "f445558a56194819a9299d5142e43ab4"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "d8c55da6bba8ee621df37aa48fd2fcbe"
server_key_iv: "ca989ec70909070ef38e5701da06653d"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.206987 25520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "f445558a56194819a9299d5142e43ab4"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "d8c55da6bba8ee621df37aa48fd2fcbe"
server_key_iv: "ca989ec70909070ef38e5701da06653d"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.214304 25520 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.007s sys 0.000s
I20260929 02:23:06.218976 25534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:06.220919 25520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260929 02:23:06.221153 25520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "f445558a56194819a9299d5142e43ab4"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "d8c55da6bba8ee621df37aa48fd2fcbe"
server_key_iv: "ca989ec70909070ef38e5701da06653d"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.221414 25520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:06.259538 25520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:06.260426 25520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:06.260686 25520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:06.279057 25520 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:35801
I20260929 02:23:06.279081 25585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:35801 every 8 connection(s)
I20260929 02:23:06.280812 25520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:06.285086 25586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:06.290138 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25520
I20260929 02:23:06.290339 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:06.290908 7848 external_mini_cluster.cc:1428] Setting key f2ef778c9182c44837d9508ea5f8d694
I20260929 02:23:06.296834 25586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: Bootstrap starting.
I20260929 02:23:06.300033 25586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:06.301196 25586 log.cc:824] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:06.304643 25586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: No bootstrap required, opened a new log
I20260929 02:23:06.305161 25589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:06.292916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.7.170.62:35801)
0929 02:23:06.293971 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:06.293983 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:06.294031 (+ 48us) server_negotiation.cc:407] Connection header received
0929 02:23:06.295892 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:06.295924 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:06.296401 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:06.296781 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:06.297372 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.298708 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.300051 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.300797 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.301009 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:06.302249 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:06.302257 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:06.302272 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:06.302958 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:06.303063 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:06.303091 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:06.303204 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:06.303428 (+ 224us) server_negotiation.cc:299] Negotiation successful
0929 02:23:06.303780 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":342,"threads_started":1}
I20260929 02:23:06.312227 25586 raft_consensus.cc:348] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } }
I20260929 02:23:06.312628 25586 raft_consensus.cc:374] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:06.312703 25586 raft_consensus.cc:729] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f445558a56194819a9299d5142e43ab4, State: Initialized, Role: FOLLOWER
I20260929 02:23:06.313299 25586 consensus_queue.cc:260] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } }
I20260929 02:23:06.313527 25586 raft_consensus.cc:388] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:06.313659 25586 raft_consensus.cc:482] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:06.313813 25586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:06.315995 25586 raft_consensus.cc:504] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } }
I20260929 02:23:06.316505 25586 leader_election.cc:302] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f445558a56194819a9299d5142e43ab4; no voters:
I20260929 02:23:06.317067 25586 leader_election.cc:288] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:06.317168 25591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:06.317899 25591 raft_consensus.cc:686] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [term 1 LEADER]: Becoming Leader. State: Replica: f445558a56194819a9299d5142e43ab4, State: Running, Role: LEADER
I20260929 02:23:06.318418 25591 consensus_queue.cc:237] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } }
I20260929 02:23:06.319201 25586 sys_catalog.cc:564] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:06.321246 25593 sys_catalog.cc:455] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f445558a56194819a9299d5142e43ab4. Latest consensus state: current_term: 1 leader_uuid: "f445558a56194819a9299d5142e43ab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } } }
I20260929 02:23:06.321162 25592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f445558a56194819a9299d5142e43ab4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f445558a56194819a9299d5142e43ab4" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 35801 } } }
I20260929 02:23:06.321574 25593 sys_catalog.cc:458] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:06.321578 25592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:06.324877 25599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:06.330221 25599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:06.339571 25599 catalog_manager.cc:1347] Generated new cluster ID: d4da85008d3347d0b23b9b990d096472
I20260929 02:23:06.339664 25599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:06.362696 25599 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:06.364538 25599 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:06.381771 25599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: Generated new TSK 0
I20260929 02:23:06.382682 25599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:06.402675 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--builtin_ntp_servers=127.7.170.20:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:06.763509 25610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:06.763768 25610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:06.763875 25610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:06.771154 25610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:06.771255 25610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:06.771394 25610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:06.780318 25610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:06.781605 25610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:06.783368 25610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:06.795822 25615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:06.795881 25616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:06.795961 25618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:06.797044 25610 server_base.cc:1034] running on GCE node
I20260929 02:23:06.797652 25610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:06.798609 25610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:06.799808 25610 hybrid_clock.cc:648] HybridClock initialized: now 1790648586799756 us; error 49 us; skew 500 ppm
I20260929 02:23:06.800123 25610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:06.802340 25610 webserver.cc:460] Webserver started at http://127.7.170.1:46607/ using document root <none> and password file <none>
I20260929 02:23:06.803099 25610 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:06.803200 25610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:06.803501 25610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:06.807260 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "1c301c8371a54542bfab69409a2f29d5"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "fb34f2464884242bbdc4ed594eda7a41"
server_key_iv: "a42f267c1b92fbf96632ca807f672040"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.808071 25610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "1c301c8371a54542bfab69409a2f29d5"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "fb34f2464884242bbdc4ed594eda7a41"
server_key_iv: "a42f267c1b92fbf96632ca807f672040"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.813272 25610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260929 02:23:06.816704 25624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:06.818045 25610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:06.818192 25610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "1c301c8371a54542bfab69409a2f29d5"
format_stamp: "Formatted at 2026-09-29 02:23:06 on dist-test-slave-8cs2"
server_key: "fb34f2464884242bbdc4ed594eda7a41"
server_key_iv: "a42f267c1b92fbf96632ca807f672040"
server_key_version: "encryptionkey@0"
I20260929 02:23:06.818392 25610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:06.832873 25610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:06.833757 25610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:06.834010 25610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:06.834949 25610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:06.836349 25610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:06.836449 25610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:06.836555 25610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:06.836621 25610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:06.865582 25610 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:33579
I20260929 02:23:06.865638 25736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:33579 every 8 connection(s)
I20260929 02:23:06.866863 25610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:06.874423 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25610
I20260929 02:23:06.874581 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:06.875016 7848 external_mini_cluster.cc:1428] Setting key d11ed86c62ae0e0197eec77364f0506b
I20260929 02:23:06.877897 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--builtin_ntp_servers=127.7.170.20:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:06.879539 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:06.869407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:34189 (local address 127.7.170.62:35801)
0929 02:23:06.869831 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:06.869835 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:06.870964 (+ 1129us) server_negotiation.cc:407] Connection header received
0929 02:23:06.872725 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:06.872731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:06.872819 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:06.872913 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:06.874038 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.874840 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.876527 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.877084 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.877186 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:06.878734 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:06.878741 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:06.878744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:06.878763 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:06.878803 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:06.878815 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:06.878888 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:06.879097 (+ 209us) server_negotiation.cc:299] Negotiation successful
0929 02:23:06.879196 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":220,"threads_started":1}
I20260929 02:23:06.880609 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:06.869888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:35801 (local address 127.7.170.1:34189)
0929 02:23:06.870784 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:06.870832 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:06.872451 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:06.873181 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:06.873190 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:06.873289 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:06.873869 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:06.873886 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.874976 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.874979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:06.876388 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:06.876402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:06.877304 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:06.877312 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:06.877617 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:06.878426 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:06.878429 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:06.878471 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:06.878552 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:06.878899 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:06.878910 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:06.878919 (+ 9us) client_negotiation.cc:769] Sending connection context
0929 02:23:06.879075 (+ 156us) client_negotiation.cc:240] Negotiation successful
0929 02:23:06.879497 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":271,"threads_started":1}
I20260929 02:23:06.882270 25737 heartbeater.cc:347] Connected to a master server at 127.7.170.62:35801
I20260929 02:23:06.882560 25737 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:06.883302 25737 heartbeater.cc:511] Master 127.7.170.62:35801 requested a full tablet report, sending...
I20260929 02:23:06.885360 25551 ts_manager.cc:194] Registered new tserver with Master: 1c301c8371a54542bfab69409a2f29d5 (127.7.170.1:33579)
I20260929 02:23:06.886803 25551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.1:34189
W20260929 02:23:07.198402 25741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:07.198681 25741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:07.198792 25741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:07.206077 25741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:07.206183 25741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:07.206314 25741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:07.215387 25741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:07.216656 25741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:07.218408 25741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:07.230921 25749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:07.230999 25746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:07.231032 25747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:07.231834 25741 server_base.cc:1034] running on GCE node
I20260929 02:23:07.232411 25741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:07.233348 25741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:07.234557 25741 hybrid_clock.cc:648] HybridClock initialized: now 1790648587234506 us; error 42 us; skew 500 ppm
I20260929 02:23:07.234855 25741 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:07.237040 25741 webserver.cc:460] Webserver started at http://127.7.170.2:38491/ using document root <none> and password file <none>
I20260929 02:23:07.237721 25741 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:07.237811 25741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:07.238114 25741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:07.240888 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "d9a02aa6c5c64928af53e017b347dbc5"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "88cddaee4b89aacc393544cc28b34f1d"
server_key_iv: "0c260fe8dba548590ee824bac17cd2c2"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.241715 25741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "d9a02aa6c5c64928af53e017b347dbc5"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "88cddaee4b89aacc393544cc28b34f1d"
server_key_iv: "0c260fe8dba548590ee824bac17cd2c2"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.246577 25741 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:07.249929 25755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.251207 25741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:07.251348 25741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "d9a02aa6c5c64928af53e017b347dbc5"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "88cddaee4b89aacc393544cc28b34f1d"
server_key_iv: "0c260fe8dba548590ee824bac17cd2c2"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.251546 25741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:07.273056 25741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:07.273895 25741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:07.274152 25741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:07.275136 25741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:07.276582 25741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:07.276664 25741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.276762 25741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:07.276836 25741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.307156 25741 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:36769
I20260929 02:23:07.307225 25867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:36769 every 8 connection(s)
I20260929 02:23:07.308393 25741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:07.313352 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25741
I20260929 02:23:07.313537 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:07.314069 7848 external_mini_cluster.cc:1428] Setting key a2e7f0c461a380e6131f6ee602996537
I20260929 02:23:07.317081 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--builtin_ntp_servers=127.7.170.20:40915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:07.321717 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.310950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:39019 (local address 127.7.170.62:35801)
0929 02:23:07.311135 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.311139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.312558 (+ 1419us) server_negotiation.cc:407] Connection header received
0929 02:23:07.314975 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.314985 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.315071 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.315168 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:07.316411 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.317451 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.318965 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.319512 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.319623 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.320908 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:07.320914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:07.320918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:07.320963 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:07.321001 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:07.321015 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:07.321094 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.321259 (+ 165us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.321359 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:07.322788 25870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.311445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:35801 (local address 127.7.170.2:39019)
0929 02:23:07.312385 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.312432 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.314680 (+ 2248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.315383 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.315392 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.315492 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:07.316244 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.316265 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.317575 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.317579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.318831 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.318843 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.319700 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.319707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.319926 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.320610 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:07.320612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:07.320652 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:07.320734 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:07.321087 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:07.321094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:07.321100 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.321237 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.321547 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":306,"threads_started":1}
I20260929 02:23:07.324163 25868 heartbeater.cc:347] Connected to a master server at 127.7.170.62:35801
I20260929 02:23:07.324492 25868 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:07.325482 25868 heartbeater.cc:511] Master 127.7.170.62:35801 requested a full tablet report, sending...
I20260929 02:23:07.327034 25551 ts_manager.cc:194] Registered new tserver with Master: d9a02aa6c5c64928af53e017b347dbc5 (127.7.170.2:36769)
I20260929 02:23:07.327919 25551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.2:39019
W20260929 02:23:07.642202 25871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:07.642469 25871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:07.642590 25871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:07.649668 25871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:07.649763 25871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:07.649904 25871 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:07.659083 25871 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:35801
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:07.660398 25871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:07.662189 25871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:07.675081 25877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:07.675233 25876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:07.675180 25879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:07.676205 25871 server_base.cc:1034] running on GCE node
I20260929 02:23:07.676865 25871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:07.677945 25871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:07.679175 25871 hybrid_clock.cc:648] HybridClock initialized: now 1790648587679122 us; error 42 us; skew 500 ppm
I20260929 02:23:07.679502 25871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:07.681870 25871 webserver.cc:460] Webserver started at http://127.7.170.3:37899/ using document root <none> and password file <none>
I20260929 02:23:07.682554 25871 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:07.682646 25871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:07.682916 25871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:07.685943 25871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "5d2c038d08594b189c666234d14be623"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "3d32995b6fc1c81f5c99da9a4d551f15"
server_key_iv: "d594809f96e9b32ca4dba48517aa7c58"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.686695 25871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "5d2c038d08594b189c666234d14be623"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "3d32995b6fc1c81f5c99da9a4d551f15"
server_key_iv: "d594809f96e9b32ca4dba48517aa7c58"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.691933 25871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:23:07.695492 25885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.697000 25871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:07.697151 25871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "5d2c038d08594b189c666234d14be623"
format_stamp: "Formatted at 2026-09-29 02:23:07 on dist-test-slave-8cs2"
server_key: "3d32995b6fc1c81f5c99da9a4d551f15"
server_key_iv: "d594809f96e9b32ca4dba48517aa7c58"
server_key_version: "encryptionkey@0"
I20260929 02:23:07.697348 25871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:07.722505 25871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:07.723376 25871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:07.723627 25871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:07.724598 25871 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:07.726217 25871 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:07.726325 25871 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.726436 25871 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:07.726511 25871 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:07.757084 25871 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:39335
I20260929 02:23:07.757159 25997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:39335 every 8 connection(s)
I20260929 02:23:07.758360 25871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:07.763702 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 25871
I20260929 02:23:07.763857 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:07.764279 7848 external_mini_cluster.cc:1428] Setting key 1718b37145ebe23576b3f0b0677f353f
I20260929 02:23:07.770642 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.760809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35405 (local address 127.7.170.62:35801)
0929 02:23:07.761053 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.761058 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.762467 (+ 1409us) server_negotiation.cc:407] Connection header received
0929 02:23:07.764322 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.764328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.764411 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.764497 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:07.765692 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.766578 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.768181 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.768576 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.768671 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.769931 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:07.769936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:07.769939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:07.769957 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:07.769987 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:07.769999 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:07.770078 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.770308 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.770391 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260929 02:23:07.771613 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.761397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:35801 (local address 127.7.170.3:35405)
0929 02:23:07.762310 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.762346 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.764071 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.764729 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.764738 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.764834 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:07.765515 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.765533 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.766742 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.766748 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.768053 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.768065 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.768749 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.768754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.768993 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.769615 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:07.769617 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:07.769657 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:07.769737 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:07.770117 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:07.770127 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:07.770133 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.770284 (+ 151us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.770581 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":276,"threads_started":1}
I20260929 02:23:07.772847 25998 heartbeater.cc:347] Connected to a master server at 127.7.170.62:35801
I20260929 02:23:07.773159 25998 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:07.773876 25998 heartbeater.cc:511] Master 127.7.170.62:35801 requested a full tablet report, sending...
I20260929 02:23:07.775202 25551 ts_manager.cc:194] Registered new tserver with Master: 5d2c038d08594b189c666234d14be623 (127.7.170.3:39335)
I20260929 02:23:07.775979 25551 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.3:35405
I20260929 02:23:07.779157 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:07.788549 25739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.783222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.7.170.62:35801)
0929 02:23:07.783426 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.783432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.783726 (+ 294us) server_negotiation.cc:407] Connection header received
0929 02:23:07.783855 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.783859 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.783958 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.784036 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:07.784428 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.785166 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.786514 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.786915 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.787001 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.787921 (+ 920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:07.787926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:07.787955 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:07.787974 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:07.788007 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:07.788019 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:07.788091 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.788183 (+ 92us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.788257 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:23:07.792271 25551 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:07.799095 25551 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:07.824541 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.811683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47004 (local address 127.7.170.3:39335)
0929 02:23:07.815272 (+ 3589us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.815282 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.815318 (+ 36us) server_negotiation.cc:407] Connection header received
0929 02:23:07.815431 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.815439 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.815689 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.816650 (+ 961us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:07.816963 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.819779 (+ 2816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.822063 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.823466 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.823598 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.823833 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.824037 (+ 204us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.824206 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2324,"thread_start_us":2162,"threads_started":1}
I20260929 02:23:07.824703 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.811380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34150)
0929 02:23:07.814426 (+ 3046us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.814446 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.814600 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.815611 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.815621 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.815653 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:07.816148 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.816161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.818291 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.818297 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.821125 (+ 2828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.821140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.823536 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.823543 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.823766 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.823843 (+ 77us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.823984 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.824274 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2921,"mutex_wait_us":40,"thread_start_us":7486,"threads_started":1}
I20260929 02:23:07.825186 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.811847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.7.170.1:33579)
0929 02:23:07.813675 (+ 1828us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.813685 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.814729 (+ 1044us) server_negotiation.cc:407] Connection header received
0929 02:23:07.814851 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.814860 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.815135 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.815326 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:07.816386 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.818117 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.821339 (+ 3222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.823083 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.823209 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.823461 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.824663 (+ 1202us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.824831 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1691,"thread_start_us":1587,"threads_started":1}
I20260929 02:23:07.825433 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.811198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:39335 (local address 127.0.0.1:47004)
0929 02:23:07.812915 (+ 1717us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.812990 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.813146 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.815917 (+ 2771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.815922 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.815950 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:07.816449 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.816461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.819930 (+ 3469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.819935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.821820 (+ 1885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.821834 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.823669 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.823675 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.823860 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.823886 (+ 26us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.823984 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.824267 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1453,"mutex_wait_us":100,"thread_start_us":1280,"threads_started":1}
I20260929 02:23:07.826638 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.812635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:36769 (local address 127.0.0.1:45452)
0929 02:23:07.819105 (+ 6470us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.819121 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.819217 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.819841 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.819846 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.819872 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:07.820320 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.820331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.821896 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.821901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.823990 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.824004 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.825944 (+ 1940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.825950 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.826107 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.826137 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.826216 (+ 79us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.826328 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6362,"mutex_wait_us":644,"thread_start_us":868,"threads_started":1}
I20260929 02:23:07.827419 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.813119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45452 (local address 127.7.170.2:36769)
0929 02:23:07.819163 (+ 6044us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.819169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.819193 (+ 24us) server_negotiation.cc:407] Connection header received
0929 02:23:07.819398 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.819406 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.819645 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.819824 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:07.820581 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.821774 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.824216 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.825662 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.825772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.825996 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.826936 (+ 940us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.827096 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5899,"thread_start_us":5780,"threads_started":1}
I20260929 02:23:07.829046 25672 tablet_service.cc:1467] Processing CreateTablet for tablet dee2b45f28204f729ae043a50e8815e6 (DEFAULT_TABLE table=test-table [id=be9b77974d5b428f9d938f39d241f4ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:07.829226 25933 tablet_service.cc:1467] Processing CreateTablet for tablet dee2b45f28204f729ae043a50e8815e6 (DEFAULT_TABLE table=test-table [id=be9b77974d5b428f9d938f39d241f4ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:07.830226 25672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee2b45f28204f729ae043a50e8815e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:07.830559 25933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee2b45f28204f729ae043a50e8815e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:07.831046 25803 tablet_service.cc:1467] Processing CreateTablet for tablet dee2b45f28204f729ae043a50e8815e6 (DEFAULT_TABLE table=test-table [id=be9b77974d5b428f9d938f39d241f4ef]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:07.832228 25803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dee2b45f28204f729ae043a50e8815e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:07.842041 26016 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Bootstrap starting.
I20260929 02:23:07.844323 26017 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Bootstrap starting.
I20260929 02:23:07.845505 26016 tablet_bootstrap.cc:654] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:07.846728 26016 log.cc:824] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:07.847707 26017 tablet_bootstrap.cc:654] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:07.848093 26018 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Bootstrap starting.
I20260929 02:23:07.849006 26017 log.cc:824] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:07.849649 26016 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: No bootstrap required, opened a new log
I20260929 02:23:07.849900 26016 ts_tablet_manager.cc:1397] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260929 02:23:07.852563 26018 tablet_bootstrap.cc:654] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:07.853564 26018 log.cc:824] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:07.855221 26017 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: No bootstrap required, opened a new log
I20260929 02:23:07.855479 26017 ts_tablet_manager.cc:1397] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260929 02:23:07.856493 26018 tablet_bootstrap.cc:492] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: No bootstrap required, opened a new log
I20260929 02:23:07.856734 26018 ts_tablet_manager.cc:1397] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260929 02:23:07.857754 26016 raft_consensus.cc:348] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.858107 26016 raft_consensus.cc:374] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:07.858201 26016 raft_consensus.cc:729] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c301c8371a54542bfab69409a2f29d5, State: Initialized, Role: FOLLOWER
I20260929 02:23:07.858741 26016 consensus_queue.cc:260] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.859726 25737 heartbeater.cc:503] Master 127.7.170.62:35801 was elected leader, sending a full tablet report...
I20260929 02:23:07.860352 26016 ts_tablet_manager.cc:1428] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260929 02:23:07.867239 26017 raft_consensus.cc:348] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.867239 26018 raft_consensus.cc:348] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.867703 26017 raft_consensus.cc:374] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:07.867703 26018 raft_consensus.cc:374] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:07.867800 26018 raft_consensus.cc:729] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9a02aa6c5c64928af53e017b347dbc5, State: Initialized, Role: FOLLOWER
I20260929 02:23:07.867839 26017 raft_consensus.cc:729] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d2c038d08594b189c666234d14be623, State: Initialized, Role: FOLLOWER
I20260929 02:23:07.868448 26017 consensus_queue.cc:260] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.868450 26018 consensus_queue.cc:260] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
W20260929 02:23:07.869014 25738 tablet.cc:2369] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:07.869549 25998 heartbeater.cc:503] Master 127.7.170.62:35801 was elected leader, sending a full tablet report...
I20260929 02:23:07.870337 26017 ts_tablet_manager.cc:1428] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260929 02:23:07.871908 26018 ts_tablet_manager.cc:1428] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
I20260929 02:23:07.872084 25868 heartbeater.cc:503] Master 127.7.170.62:35801 was elected leader, sending a full tablet report...
I20260929 02:23:07.932878 26024 raft_consensus.cc:482] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:07.933256 26024 raft_consensus.cc:504] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.935047 26024 leader_election.cc:288] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5d2c038d08594b189c666234d14be623 (127.7.170.3:39335), 1c301c8371a54542bfab69409a2f29d5 (127.7.170.1:33579)
I20260929 02:23:07.943966 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.935614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:37187 (local address 127.7.170.3:39335)
0929 02:23:07.935977 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.935983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.936285 (+ 302us) server_negotiation.cc:407] Connection header received
0929 02:23:07.936471 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.936475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.936538 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.936609 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:07.937672 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.938748 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.941573 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.942828 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.942949 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.943041 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.943570 (+ 529us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.943709 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260929 02:23:07.943966 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.935569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:43015 (local address 127.7.170.1:33579)
0929 02:23:07.935820 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:07.935827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:07.936522 (+ 695us) server_negotiation.cc:407] Connection header received
0929 02:23:07.936660 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:07.936663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:07.936734 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:07.936807 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:07.937555 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.938642 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.941584 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.942805 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.942941 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.943041 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:07.943570 (+ 529us) server_negotiation.cc:299] Negotiation successful
0929 02:23:07.943709 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260929 02:23:07.944154 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.935401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.7.170.2:43015)
0929 02:23:07.936420 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.936435 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.936543 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.936834 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.936838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.936861 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:07.937372 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.937385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.938769 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.938774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.941343 (+ 2569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.941359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.943043 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.943053 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.943329 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.943408 (+ 79us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.943506 (+ 98us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.943726 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":902,"mutex_wait_us":284,"thread_start_us":376,"threads_started":1}
I20260929 02:23:07.944425 26026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:07.935288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:39335 (local address 127.7.170.2:37187)
0929 02:23:07.936153 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:07.936176 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:07.936342 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:07.936995 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:07.937000 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:07.937095 (+ 95us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:07.937512 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.937522 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.938882 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.938886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.941342 (+ 2456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:07.941359 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:07.943065 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:07.943071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:07.943328 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:07.943405 (+ 77us) client_negotiation.cc:769] Sending connection context
0929 02:23:07.943502 (+ 97us) client_negotiation.cc:240] Negotiation successful
0929 02:23:07.943727 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":243,"spinlock_wait_cycles":28416,"thread_start_us":826,"threads_started":1}
I20260929 02:23:07.945452 25692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee2b45f28204f729ae043a50e8815e6" candidate_uuid: "d9a02aa6c5c64928af53e017b347dbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c301c8371a54542bfab69409a2f29d5" is_pre_election: true
I20260929 02:23:07.945452 25953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee2b45f28204f729ae043a50e8815e6" candidate_uuid: "d9a02aa6c5c64928af53e017b347dbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d2c038d08594b189c666234d14be623" is_pre_election: true
I20260929 02:23:07.946076 25953 raft_consensus.cc:2454] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a02aa6c5c64928af53e017b347dbc5 in term 0.
I20260929 02:23:07.946076 25692 raft_consensus.cc:2454] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9a02aa6c5c64928af53e017b347dbc5 in term 0.
I20260929 02:23:07.946821 25756 leader_election.cc:302] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c301c8371a54542bfab69409a2f29d5, 5d2c038d08594b189c666234d14be623, d9a02aa6c5c64928af53e017b347dbc5; no voters:
I20260929 02:23:07.947263 26024 raft_consensus.cc:2781] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:07.947410 26024 raft_consensus.cc:482] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:07.947499 26024 raft_consensus.cc:3037] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:07.949713 26024 raft_consensus.cc:504] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.950266 26024 leader_election.cc:288] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [CANDIDATE]: Term 1 election: Requested vote from peers 5d2c038d08594b189c666234d14be623 (127.7.170.3:39335), 1c301c8371a54542bfab69409a2f29d5 (127.7.170.1:33579)
I20260929 02:23:07.950814 25692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee2b45f28204f729ae043a50e8815e6" candidate_uuid: "d9a02aa6c5c64928af53e017b347dbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c301c8371a54542bfab69409a2f29d5"
I20260929 02:23:07.950815 25953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dee2b45f28204f729ae043a50e8815e6" candidate_uuid: "d9a02aa6c5c64928af53e017b347dbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d2c038d08594b189c666234d14be623"
I20260929 02:23:07.951030 25953 raft_consensus.cc:3037] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:07.951031 25692 raft_consensus.cc:3037] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:07.954229 25953 raft_consensus.cc:2454] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a02aa6c5c64928af53e017b347dbc5 in term 1.
I20260929 02:23:07.954224 25692 raft_consensus.cc:2454] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9a02aa6c5c64928af53e017b347dbc5 in term 1.
I20260929 02:23:07.954691 25759 leader_election.cc:302] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c301c8371a54542bfab69409a2f29d5, 5d2c038d08594b189c666234d14be623, d9a02aa6c5c64928af53e017b347dbc5; no voters:
I20260929 02:23:07.955053 26024 raft_consensus.cc:2781] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:07.955471 26024 raft_consensus.cc:686] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 1 LEADER]: Becoming Leader. State: Replica: d9a02aa6c5c64928af53e017b347dbc5, State: Running, Role: LEADER
I20260929 02:23:07.955935 26024 consensus_queue.cc:237] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } }
I20260929 02:23:07.960675 25549 catalog_manager.cc:5495] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d9a02aa6c5c64928af53e017b347dbc5 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "d9a02aa6c5c64928af53e017b347dbc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9a02aa6c5c64928af53e017b347dbc5" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36769 } health_report { overall_health: HEALTHY } } }
W20260929 02:23:08.010068 25999 tablet.cc:2369] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:08.013592 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.007816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45462 (local address 127.7.170.2:36769)
0929 02:23:08.008013 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:08.008018 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:08.008104 (+ 86us) server_negotiation.cc:407] Connection header received
0929 02:23:08.008288 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:08.008292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:08.008412 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:08.008540 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:08.008979 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.009681 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.011437 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.011873 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.011965 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:08.012481 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:08.013034 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:08.013143 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:08.013258 (+ 115us) server_negotiation.cc:299] Negotiation successful
0929 02:23:08.013325 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:23:08.022526 25692 raft_consensus.cc:1261] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 1 FOLLOWER]: Refusing update from remote peer d9a02aa6c5c64928af53e017b347dbc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:08.022537 25953 raft_consensus.cc:1261] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 1 FOLLOWER]: Refusing update from remote peer d9a02aa6c5c64928af53e017b347dbc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:08.023700 26027 consensus_queue.cc:1035] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c301c8371a54542bfab69409a2f29d5" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 33579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:08.024088 26024 consensus_queue.cc:1035] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d2c038d08594b189c666234d14be623" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:08.051335 25549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:08.051600 25549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:08.055047 25549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f445558a56194819a9299d5142e43ab4: Sending DeleteTablet for 3 replicas of tablet dee2b45f28204f729ae043a50e8815e6
I20260929 02:23:08.056263 25803 tablet_service.cc:1514] Processing DeleteTablet for tablet dee2b45f28204f729ae043a50e8815e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:08 UTC) from {username='slave'} at 127.0.0.1:45452
I20260929 02:23:08.056238 25933 tablet_service.cc:1514] Processing DeleteTablet for tablet dee2b45f28204f729ae043a50e8815e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:08 UTC) from {username='slave'} at 127.0.0.1:47004
I20260929 02:23:08.056294 25672 tablet_service.cc:1514] Processing DeleteTablet for tablet dee2b45f28204f729ae043a50e8815e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:08 UTC) from {username='slave'} at 127.0.0.1:34150
I20260929 02:23:08.057744 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25610
I20260929 02:23:08.091373 26041 tablet_replica.cc:330] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: stopping tablet replica
I20260929 02:23:08.091964 26041 raft_consensus.cc:2229] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:08.092751 26041 raft_consensus.cc:2258] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:08.093606 26040 tablet_replica.cc:330] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: stopping tablet replica
I20260929 02:23:08.094053 26040 raft_consensus.cc:2229] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:08.094583 26040 raft_consensus.cc:2258] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:08.095736 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.059824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.7.170.1:33579)
0929 02:23:08.060019 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:08.060027 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:08.082919 (+ 22892us) server_negotiation.cc:407] Connection header received
0929 02:23:08.083509 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:08.083517 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:08.083714 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:08.083801 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:08.084560 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.085964 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.088372 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.089142 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.089248 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:08.093906 (+ 4658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:08.093919 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:08.093937 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:08.094774 (+ 837us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:08.094837 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:08.094866 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:08.094961 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:08.095230 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 02:23:08.095360 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:23:08.096751 26042 tablet_replica.cc:330] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5: stopping tablet replica
I20260929 02:23:08.097012 25732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:08.097916 26040 ts_tablet_manager.cc:1905] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:08.099116 26041 ts_tablet_manager.cc:1905] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:08.105917 26040 ts_tablet_manager.cc:1918] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:08.106108 26040 log.cc:1196] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/dee2b45f28204f729ae043a50e8815e6
I20260929 02:23:08.106428 26041 ts_tablet_manager.cc:1918] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:08.106600 26041 log.cc:1196] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/dee2b45f28204f729ae043a50e8815e6
I20260929 02:23:08.106673 26040 ts_tablet_manager.cc:1939] T dee2b45f28204f729ae043a50e8815e6 P 5d2c038d08594b189c666234d14be623: Deleting consensus metadata
I20260929 02:23:08.107146 26041 ts_tablet_manager.cc:1939] T dee2b45f28204f729ae043a50e8815e6 P d9a02aa6c5c64928af53e017b347dbc5: Deleting consensus metadata
I20260929 02:23:08.108784 25538 catalog_manager.cc:4841] TS 5d2c038d08594b189c666234d14be623 (127.7.170.3:39335): tablet dee2b45f28204f729ae043a50e8815e6 (table test-table [id=be9b77974d5b428f9d938f39d241f4ef]) successfully deleted
I20260929 02:23:08.108810 25537 catalog_manager.cc:4841] TS d9a02aa6c5c64928af53e017b347dbc5 (127.7.170.2:36769): tablet dee2b45f28204f729ae043a50e8815e6 (table test-table [id=be9b77974d5b428f9d938f39d241f4ef]) successfully deleted
I20260929 02:23:08.097203 26042 raft_consensus.cc:2229] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:08.195360 26042 raft_consensus.cc:2258] T dee2b45f28204f729ae043a50e8815e6 P 1c301c8371a54542bfab69409a2f29d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:08.196781 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25610
I20260929 02:23:08.211650 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25741
W20260929 02:23:08.211727 25535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.170.1:33579 (error 108)
I20260929 02:23:08.213701 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.213232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34176)
0929 02:23:08.213392 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.213521 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260929 02:23:08.214879 25535 catalog_manager.cc:4584] TS 1c301c8371a54542bfab69409a2f29d5 (127.7.170.1:33579): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dee2b45f28204f729ae043a50e8815e6: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
I20260929 02:23:08.219794 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.212784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.7.170.2:36769)
0929 02:23:08.212981 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:08.212987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:08.213007 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:23:08.213095 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:08.213101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:08.213198 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:08.213316 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:08.213904 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.214666 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.216587 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.217113 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.217225 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:08.218354 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:08.218363 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:08.218377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:08.218989 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:08.219043 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:08.219062 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:08.219140 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:08.219295 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 02:23:08.219441 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:23:08.220485 25863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:08.268630 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.268157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34180)
0929 02:23:08.268362 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.268455 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 02:23:08.269635 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.269313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34196)
0929 02:23:08.269444 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.269518 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:23:08.330164 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.329707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34212)
0929 02:23:08.329892 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.330015 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 02:23:08.331269 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.330854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34228)
0929 02:23:08.331053 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.331130 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260929 02:23:08.332441 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25741
I20260929 02:23:08.347416 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25871
I20260929 02:23:08.354353 26012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.348240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47010 (local address 127.7.170.3:39335)
0929 02:23:08.348497 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:08.348502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:08.348523 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:23:08.348647 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:08.348652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:08.348772 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:08.348905 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:08.349333 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.350101 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.351444 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:08.351830 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:08.351943 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:08.353022 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:08.353032 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:08.353047 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:08.353628 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:08.353676 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:08.353696 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:08.353799 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:08.353922 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 02:23:08.354062 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260929 02:23:08.355014 25993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:08.411854 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.411253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34234)
0929 02:23:08.411521 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.411638 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260929 02:23:08.412967 26011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:08.412490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:33579 (local address 127.0.0.1:34248)
0929 02:23:08.412697 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:08.412800 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260929 02:23:08.441988 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25871
I20260929 02:23:08.455582 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 25520
I20260929 02:23:08.456600 25581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:08.563130 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 25520
2026-09-29T02:23:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2854 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:08 dist-test-slave-8cs2 krb5kdc[26051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:08 dist-test-slave-8cs2 krb5kdc[26051](info): set up 2 sockets
Sep 29 02:23:08 dist-test-slave-8cs2 krb5kdc[26051](info): commencing operation
krb5kdc: starting...
W20260929 02:23:09.589108 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.985s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:09 dist-test-slave-8cs2 krb5kdc[26051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:09.842026 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42379
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:10.159909 26064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:10.160163 26064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:10.160279 26064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:10.167440 26064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:10.167537 26064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:10.167606 26064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:10.167680 26064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:10.167737 26064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:10.177232 26064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:10.178455 26064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:10.179703 26064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:10.190673 26069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:10.190639 26070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:10.190600 26072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:10.191437 26064 server_base.cc:1034] running on GCE node
I20260929 02:23:10.192162 26064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:10.193569 26064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:10.194796 26064 hybrid_clock.cc:648] HybridClock initialized: now 1790648590194725 us; error 41 us; skew 500 ppm
I20260929 02:23:10.195181 26064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:10 dist-test-slave-8cs2 krb5kdc[26051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:10.198208 26064 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:10.199682 26064 webserver.cc:460] Webserver started at http://127.7.170.62:42145/ using document root <none> and password file <none>
I20260929 02:23:10.200377 26064 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:10.200495 26064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:10.200783 26064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:10.203811 26064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "57514b2345a34d3788072bcf8ca4ca24"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "adcb623c5d4da155450a9f57b487d89c"
server_key_iv: "0aff758fe916754a774beb54085f3155"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.204521 26064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "57514b2345a34d3788072bcf8ca4ca24"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "adcb623c5d4da155450a9f57b487d89c"
server_key_iv: "0aff758fe916754a774beb54085f3155"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.209509 26064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260929 02:23:10.212807 26079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:10.214152 26064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:10.214314 26064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "57514b2345a34d3788072bcf8ca4ca24"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "adcb623c5d4da155450a9f57b487d89c"
server_key_iv: "0aff758fe916754a774beb54085f3155"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.214491 26064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:10.231624 26064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:10.235563 26064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:10.235826 26064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:10.252509 26064 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42379
I20260929 02:23:10.252532 26130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42379 every 8 connection(s)
I20260929 02:23:10.253893 26064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:10.254830 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26064
I20260929 02:23:10.255005 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:10.255399 7848 external_mini_cluster.cc:1428] Setting key 87e1481677678b7f6f20b57d9eadf2b6
I20260929 02:23:10.257949 26131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:23:10 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:10.270155 26131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: Bootstrap starting.
I20260929 02:23:10.274052 26131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:10.275174 26131 log.cc:824] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:10.278337 26131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: No bootstrap required, opened a new log
I20260929 02:23:10.278542 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:10.257096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46534 (local address 127.7.170.62:42379)
0929 02:23:10.257816 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:10.257827 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:10.257867 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:23:10.259451 (+ 1584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:10.259481 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:10.259839 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:10.260143 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:10.261814 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.263238 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.265087 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.265749 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.265897 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:10.270572 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:10.270583 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:10.270602 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:10.270647 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:10.274240 (+ 3593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:10.274947 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:10.274957 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:10.274970 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:10.275169 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:10.275580 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:10.275584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:10.275588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:10.276168 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:10.276417 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:10.276849 (+ 432us) server_negotiation.cc:299] Negotiation successful
0929 02:23:10.277297 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":328,"threads_started":1}
I20260929 02:23:10.285501 26131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } }
I20260929 02:23:10.285843 26131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:10.285910 26131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57514b2345a34d3788072bcf8ca4ca24, State: Initialized, Role: FOLLOWER
I20260929 02:23:10.286362 26131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } }
I20260929 02:23:10.286551 26131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:10.286639 26131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:10.286758 26131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:10.288759 26131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } }
I20260929 02:23:10.289247 26131 leader_election.cc:302] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57514b2345a34d3788072bcf8ca4ca24; no voters:
I20260929 02:23:10.289776 26131 leader_election.cc:288] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:10.289942 26136 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:10.290881 26136 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [term 1 LEADER]: Becoming Leader. State: Replica: 57514b2345a34d3788072bcf8ca4ca24, State: Running, Role: LEADER
I20260929 02:23:10.291352 26136 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } }
I20260929 02:23:10.291739 26131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:10.293890 26138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57514b2345a34d3788072bcf8ca4ca24. Latest consensus state: current_term: 1 leader_uuid: "57514b2345a34d3788072bcf8ca4ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } } }
I20260929 02:23:10.294288 26138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:10.294184 26137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57514b2345a34d3788072bcf8ca4ca24" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57514b2345a34d3788072bcf8ca4ca24" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42379 } } }
I20260929 02:23:10.294466 26137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:10.295928 26145 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:10.300241 26145 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:10.308629 26145 catalog_manager.cc:1347] Generated new cluster ID: 8203ac726d9743ef82a640bbc09b0d22
I20260929 02:23:10.308751 26145 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:10.316962 26145 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:10.318869 26145 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:10.327704 26145 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: Generated new TSK 0
I20260929 02:23:10.328565 26145 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:23:10.399696 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:10.727149 26157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:10.727434 26157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:10.727587 26157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:10.736816 26157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:10.737031 26157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:10.737237 26157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:10.747993 26157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:10.749382 26157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:10.750928 26157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:10.763556 26162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:10.763603 26163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:10.763530 26165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:10.764366 26157 server_base.cc:1034] running on GCE node
I20260929 02:23:10.765015 26157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:10.765995 26157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:10.767220 26157 hybrid_clock.cc:648] HybridClock initialized: now 1790648590767171 us; error 42 us; skew 500 ppm
I20260929 02:23:10.767547 26157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:10 dist-test-slave-8cs2 krb5kdc[26051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:10.770648 26157 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:23:10.772008 26157 webserver.cc:460] Webserver started at http://127.7.170.1:38931/ using document root <none> and password file <none>
I20260929 02:23:10.772639 26157 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:10.772732 26157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:10.773056 26157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:10.775842 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "8efed5199d67474595568931335aad9d"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "d3b24a5a6b132e6e177e0c648a965251"
server_key_iv: "594453df20bf5e5e6f70dcb8dcc22548"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.776638 26157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "8efed5199d67474595568931335aad9d"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "d3b24a5a6b132e6e177e0c648a965251"
server_key_iv: "594453df20bf5e5e6f70dcb8dcc22548"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.781627 26157 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260929 02:23:10.785156 26172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:10.786491 26157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260929 02:23:10.786636 26157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "8efed5199d67474595568931335aad9d"
format_stamp: "Formatted at 2026-09-29 02:23:10 on dist-test-slave-8cs2"
server_key: "d3b24a5a6b132e6e177e0c648a965251"
server_key_iv: "594453df20bf5e5e6f70dcb8dcc22548"
server_key_version: "encryptionkey@0"
I20260929 02:23:10.786821 26157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:10.799126 26157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:10.802928 26157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:10.803185 26157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:10.804105 26157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:10.805469 26157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:10.805543 26157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:10.805648 26157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:10.805698 26157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:10.834968 26157 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:39915
I20260929 02:23:10.835033 26284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:39915 every 8 connection(s)
I20260929 02:23:10.836210 26157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:10.837745 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26157
I20260929 02:23:10.837930 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:10.838444 7848 external_mini_cluster.cc:1428] Setting key f9986070413904443d54264ea0bc787b
Sep 29 02:23:10 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648590, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:10.855506 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:10.838902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:47901 (local address 127.7.170.62:42379)
0929 02:23:10.839365 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:10.839371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:10.840414 (+ 1043us) server_negotiation.cc:407] Connection header received
0929 02:23:10.842286 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:10.842291 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:10.842388 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:10.842479 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:10.843976 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.844690 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.846257 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.846651 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.846763 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:10.850195 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:10.850199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:10.850202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:10.850233 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:10.852577 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:10.853316 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:10.853320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:10.853322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:10.853447 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:10.853835 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:10.853839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:10.853841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:10.854091 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:10.854244 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:10.854923 (+ 679us) server_negotiation.cc:299] Negotiation successful
0929 02:23:10.855146 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":231,"threads_started":1}
I20260929 02:23:10.856403 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:10.839481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42379 (local address 127.7.170.1:47901)
0929 02:23:10.840286 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:10.840324 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:10.842036 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:10.842715 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:10.842723 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:10.843239 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:10.843813 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:10.843830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.844835 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.844838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:10.846121 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:10.846133 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:10.846876 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:10.846883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:10.847103 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:10.847706 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:10.847729 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:10.849993 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:10.852704 (+ 2711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:10.852714 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:10.852731 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:10.853200 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:10.853544 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:10.853549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:10.853553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:10.853724 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:10.854210 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:10.854217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:10.854559 (+ 342us) client_negotiation.cc:769] Sending connection context
0929 02:23:10.854791 (+ 232us) client_negotiation.cc:240] Negotiation successful
0929 02:23:10.855318 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":328,"threads_started":1}
I20260929 02:23:10.857862 26285 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42379
I20260929 02:23:10.858201 26285 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:10.859084 26285 heartbeater.cc:511] Master 127.7.170.62:42379 requested a full tablet report, sending...
I20260929 02:23:10.861493 26096 ts_manager.cc:194] Registered new tserver with Master: 8efed5199d67474595568931335aad9d (127.7.170.1:39915)
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
I20260929 02:23:10.863009 26096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:47901
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:23:10.895906 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:11.217051 26291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:11.217365 26291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:11.217496 26291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:11.226248 26291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:11.226410 26291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:11.226574 26291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:11.237705 26291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:11.239394 26291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:11.241101 26291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:11.255398 26297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:11.255463 26299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:11.256006 26296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:11.256564 26291 server_base.cc:1034] running on GCE node
I20260929 02:23:11.257346 26291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:11.258652 26291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:11.259881 26291 hybrid_clock.cc:648] HybridClock initialized: now 1790648591259818 us; error 47 us; skew 500 ppm
I20260929 02:23:11.260294 26291 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:11 dist-test-slave-8cs2 krb5kdc[26051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:11.271090 26291 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:23:11.272862 26291 webserver.cc:460] Webserver started at http://127.7.170.2:43097/ using document root <none> and password file <none>
I20260929 02:23:11.273772 26291 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:11.273890 26291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:11.274242 26291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:11.277814 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "2445a10c716cd5d6a851003724f192f2"
server_key_iv: "04427a9824dc5f4d91b05ca75f3e7d53"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.278702 26291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "2445a10c716cd5d6a851003724f192f2"
server_key_iv: "04427a9824dc5f4d91b05ca75f3e7d53"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.285578 26291 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260929 02:23:11.290947 26306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.292908 26291 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:23:11.293136 26291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "2445a10c716cd5d6a851003724f192f2"
server_key_iv: "04427a9824dc5f4d91b05ca75f3e7d53"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.293355 26291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:11.327072 26291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:11.331638 26291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:11.331912 26291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:11.333009 26291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:11.334515 26291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:11.334590 26291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.334703 26291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:11.334746 26291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.368727 26291 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:41841
I20260929 02:23:11.368784 26418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:41841 every 8 connection(s)
I20260929 02:23:11.370421 26291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:11.371942 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26291
I20260929 02:23:11.372169 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:11.372735 7848 external_mini_cluster.cc:1428] Setting key 0e6f8b265b46fffc827b2a1d0edbb8d8
Sep 29 02:23:11 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:11.396730 26421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:11.373818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:51161 (local address 127.7.170.62:42379)
0929 02:23:11.374321 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:11.374326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:11.375885 (+ 1559us) server_negotiation.cc:407] Connection header received
0929 02:23:11.378775 (+ 2890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:11.378781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:11.378949 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:11.379070 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:11.381412 (+ 2342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.382256 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.384499 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.385112 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.385239 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:11.389680 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:11.389687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:11.389690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:11.389735 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:11.392917 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:11.393780 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:11.393786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:11.393788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:11.393955 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:11.394512 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:11.394516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:11.394518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:11.394894 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:11.395050 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:11.395958 (+ 908us) server_negotiation.cc:299] Negotiation successful
0929 02:23:11.396233 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":257,"threads_started":1}
I20260929 02:23:11.398268 26422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:11.374688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42379 (local address 127.7.170.2:51161)
0929 02:23:11.375674 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:11.375741 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:11.378381 (+ 2640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:11.379407 (+ 1026us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:11.379420 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:11.380259 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:11.381227 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:11.381256 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.382406 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.382410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:11.384324 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:11.384338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.385358 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.385364 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:11.385672 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:11.386507 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:11.386541 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:11.389379 (+ 2838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:11.393097 (+ 3718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:11.393109 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:11.393132 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:11.393638 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:11.394078 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:11.394084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:11.394088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:11.394373 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:11.395028 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:11.395048 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:11.395511 (+ 463us) client_negotiation.cc:769] Sending connection context
0929 02:23:11.395827 (+ 316us) client_negotiation.cc:240] Negotiation successful
0929 02:23:11.396502 (+ 675us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":401,"threads_started":1}
I20260929 02:23:11.400239 26419 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42379
I20260929 02:23:11.400614 26419 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:11.401631 26419 heartbeater.cc:511] Master 127.7.170.62:42379 requested a full tablet report, sending...
I20260929 02:23:11.403527 26096 ts_manager.cc:194] Registered new tserver with Master: e0e4f4c401bb4c22853cdec1f9d6a2f8 (127.7.170.2:41841)
I20260929 02:23:11.404875 26096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:51161
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:23:11.443681 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:39091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:11.817898 26425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:11.818231 26425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:11.818369 26425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:11.825917 26425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:11.826025 26425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:11.826179 26425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:11.836483 26425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:39091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:42379
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:11.838047 26425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:11.839568 26425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:11.854336 26430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:11.854317 26431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:11.854681 26425 server_base.cc:1034] running on GCE node
W20260929 02:23:11.854918 26433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:11.855623 26425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:11.856693 26425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:11.857900 26425 hybrid_clock.cc:648] HybridClock initialized: now 1790648591857851 us; error 37 us; skew 500 ppm
I20260929 02:23:11.858253 26425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:11 dist-test-slave-8cs2 krb5kdc[26051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:11.861717 26425 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:23:11.863328 26425 webserver.cc:460] Webserver started at http://127.7.170.3:40039/ using document root <none> and password file <none>
I20260929 02:23:11.864117 26425 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:11.864223 26425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:11.864522 26425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:11.866123 26285 heartbeater.cc:503] Master 127.7.170.62:42379 was elected leader, sending a full tablet report...
I20260929 02:23:11.868207 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "095f7ed737a848d0b9b4e9189ff2772c"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "cad9f68cbc96eb9edd22750a601bab20"
server_key_iv: "85c19f0d6ab7c148a183e847b34666eb"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.869139 26425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "095f7ed737a848d0b9b4e9189ff2772c"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "cad9f68cbc96eb9edd22750a601bab20"
server_key_iv: "85c19f0d6ab7c148a183e847b34666eb"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.875174 26425 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260929 02:23:11.879369 26440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.880991 26425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:11.881160 26425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "095f7ed737a848d0b9b4e9189ff2772c"
format_stamp: "Formatted at 2026-09-29 02:23:11 on dist-test-slave-8cs2"
server_key: "cad9f68cbc96eb9edd22750a601bab20"
server_key_iv: "85c19f0d6ab7c148a183e847b34666eb"
server_key_version: "encryptionkey@0"
I20260929 02:23:11.881366 26425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:11.902483 26425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:11.907791 26425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:11.908123 26425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:11.909324 26425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:11.910894 26425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:11.910969 26425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.911085 26425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:11.911139 26425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:11.944785 26425 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:37049
I20260929 02:23:11.944854 26552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:37049 every 8 connection(s)
I20260929 02:23:11.946393 26425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:11.954632 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26425
I20260929 02:23:11.954836 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:11.955333 7848 external_mini_cluster.cc:1428] Setting key e0f3dca696bcc1b4f7085f204a31810a
Sep 29 02:23:11 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648591, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:11.969022 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:11.949481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:35109 (local address 127.7.170.62:42379)
0929 02:23:11.950000 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:11.950007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:11.951264 (+ 1257us) server_negotiation.cc:407] Connection header received
0929 02:23:11.953511 (+ 2247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:11.953517 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:11.953611 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:11.953718 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:11.955544 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.956551 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.958661 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.959113 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.959213 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:11.962910 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:11.962916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:11.962920 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:11.962959 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:11.965806 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:11.966589 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:11.966594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:11.966596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:11.966739 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:11.967236 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:11.967240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:11.967242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:11.967551 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:11.967694 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:11.968422 (+ 728us) server_negotiation.cc:299] Negotiation successful
0929 02:23:11.968631 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":214,"threads_started":1}
I20260929 02:23:11.970357 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:11.950247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:42379 (local address 127.7.170.3:35109)
0929 02:23:11.951070 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:11.951121 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:11.953207 (+ 2086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:11.953964 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:11.953974 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:11.954607 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:11.955354 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:11.955375 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.956730 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.956735 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:11.958443 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:11.958456 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.959320 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.959324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:11.959540 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:11.960249 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:11.960281 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:11.962681 (+ 2400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:11.965941 (+ 3260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:11.965949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:11.965965 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:11.966450 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:11.966866 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:11.966871 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:11.966873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:11.967088 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:11.967647 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:11.967654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:11.968019 (+ 365us) client_negotiation.cc:769] Sending connection context
0929 02:23:11.968282 (+ 263us) client_negotiation.cc:240] Negotiation successful
0929 02:23:11.968788 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":320,"threads_started":1}
I20260929 02:23:11.971962 26553 heartbeater.cc:347] Connected to a master server at 127.7.170.62:42379
I20260929 02:23:11.972358 26553 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:11.973279 26553 heartbeater.cc:511] Master 127.7.170.62:42379 requested a full tablet report, sending...
I20260929 02:23:11.975126 26096 ts_manager.cc:194] Registered new tserver with Master: 095f7ed737a848d0b9b4e9189ff2772c (127.7.170.3:37049)
I20260929 02:23:11.976178 26096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:35109
I20260929 02:23:11.983968 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:12.002861 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:11.988686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59404 (local address 127.7.170.62:42379)
0929 02:23:11.988867 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:11.988872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:11.989250 (+ 378us) server_negotiation.cc:407] Connection header received
0929 02:23:11.989455 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:11.989460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:11.989547 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:11.989651 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:11.991250 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.992185 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.993622 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:11.994097 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:11.994219 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:11.997070 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:11.997075 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:11.997079 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:11.997127 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:11.999844 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.000726 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.000730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.000733 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.000892 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.001366 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.001370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.001373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.001811 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:12.002042 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.002413 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.002594 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:23:12.007669 26096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:12.016661 26096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:12.057502 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.030606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:41841 (local address 127.0.0.1:43290)
0929 02:23:12.031503 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:12.031552 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:12.031747 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:12.047794 (+ 16047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:12.047806 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:12.047861 (+ 55us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:12.048466 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.048484 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.051491 (+ 3007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.051498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.054195 (+ 2697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.054216 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.056442 (+ 2226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.056448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.056655 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.056717 (+ 62us) client_negotiation.cc:769] Sending connection context
0929 02:23:12.056852 (+ 135us) client_negotiation.cc:240] Negotiation successful
0929 02:23:12.057131 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":464,"threads_started":1}
I20260929 02:23:12.058280 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.030793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.7.170.2:41841)
0929 02:23:12.046692 (+ 15899us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.046706 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.046760 (+ 54us) server_negotiation.cc:407] Connection header received
0929 02:23:12.046941 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.046952 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.047353 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.048763 (+ 1410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:12.049070 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.050937 (+ 1867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.054457 (+ 3520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.055896 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.056014 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.056274 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.057746 (+ 1472us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.057903 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15685,"thread_start_us":15522,"threads_started":1}
I20260929 02:23:12.061007 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.032432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36822 (local address 127.7.170.1:39915)
0929 02:23:12.052564 (+ 20132us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.052575 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.052612 (+ 37us) server_negotiation.cc:407] Connection header received
0929 02:23:12.052747 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.052757 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.053131 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.053372 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:12.054563 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.056059 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.058512 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.059883 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.060021 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.060284 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.060462 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.060631 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19907,"thread_start_us":19700,"threads_started":1}
I20260929 02:23:12.061069 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.031219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:39915 (local address 127.0.0.1:36822)
0929 02:23:12.051683 (+ 20464us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:12.051712 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:12.051865 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:12.053675 (+ 1810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:12.053685 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:12.053717 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:12.054373 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.054388 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.056252 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.056257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.058323 (+ 2066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.058335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.060110 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.060116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.060285 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.060318 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:23:12.060407 (+ 89us) client_negotiation.cc:240] Negotiation successful
0929 02:23:12.060532 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20271,"mutex_wait_us":4,"thread_start_us":22889,"threads_started":1}
I20260929 02:23:12.063010 26568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.032775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:37049 (local address 127.0.0.1:43540)
0929 02:23:12.054453 (+ 21678us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:12.054471 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:12.054604 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:12.055233 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:12.055238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:12.055267 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:12.055640 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.055651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.057970 (+ 2319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.057976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.059614 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.059626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.062265 (+ 2639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.062272 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.062433 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.062466 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:23:12.062549 (+ 83us) client_negotiation.cc:240] Negotiation successful
0929 02:23:12.062677 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21548,"thread_start_us":18522,"threads_started":1}
I20260929 02:23:12.063169 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.032378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43540 (local address 127.7.170.3:37049)
0929 02:23:12.048984 (+ 16606us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.048995 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.054591 (+ 5596us) server_negotiation.cc:407] Connection header received
0929 02:23:12.054748 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.054757 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.055030 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.055244 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:12.055811 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.057491 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.059818 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.062075 (+ 2257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.062192 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.062438 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.062616 (+ 178us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.062790 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16396,"thread_start_us":16004,"threads_started":1}
I20260929 02:23:12.063638 26354 tablet_service.cc:1467] Processing CreateTablet for tablet 835414268ec94006ada69450d382bb30 (DEFAULT_TABLE table=test-table [id=e364e41b786e41f5ba88eafa397f9e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:12.065510 26354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835414268ec94006ada69450d382bb30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:12.065373 26220 tablet_service.cc:1467] Processing CreateTablet for tablet 835414268ec94006ada69450d382bb30 (DEFAULT_TABLE table=test-table [id=e364e41b786e41f5ba88eafa397f9e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:12.066924 26220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835414268ec94006ada69450d382bb30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:12.071565 26488 tablet_service.cc:1467] Processing CreateTablet for tablet 835414268ec94006ada69450d382bb30 (DEFAULT_TABLE table=test-table [id=e364e41b786e41f5ba88eafa397f9e33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:12.072788 26488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 835414268ec94006ada69450d382bb30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:12.081490 26572 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Bootstrap starting.
I20260929 02:23:12.085745 26574 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Bootstrap starting.
I20260929 02:23:12.086861 26573 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Bootstrap starting.
I20260929 02:23:12.087306 26572 tablet_bootstrap.cc:654] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:12.088893 26572 log.cc:824] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:12.089428 26574 tablet_bootstrap.cc:654] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:12.090631 26573 tablet_bootstrap.cc:654] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:12.090921 26574 log.cc:824] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:12.091898 26573 log.cc:824] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:12.093299 26572 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: No bootstrap required, opened a new log
I20260929 02:23:12.093660 26572 ts_tablet_manager.cc:1397] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20260929 02:23:12.094039 26574 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: No bootstrap required, opened a new log
I20260929 02:23:12.094296 26574 ts_tablet_manager.cc:1397] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260929 02:23:12.100102 26573 tablet_bootstrap.cc:492] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: No bootstrap required, opened a new log
I20260929 02:23:12.100415 26573 ts_tablet_manager.cc:1397] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20260929 02:23:12.102150 26574 raft_consensus.cc:348] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.102533 26574 raft_consensus.cc:374] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:12.102643 26574 raft_consensus.cc:729] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095f7ed737a848d0b9b4e9189ff2772c, State: Initialized, Role: FOLLOWER
I20260929 02:23:12.103368 26574 consensus_queue.cc:260] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.104528 26572 raft_consensus.cc:348] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.105064 26572 raft_consensus.cc:374] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:12.105209 26572 raft_consensus.cc:729] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0e4f4c401bb4c22853cdec1f9d6a2f8, State: Initialized, Role: FOLLOWER
I20260929 02:23:12.105396 26553 heartbeater.cc:503] Master 127.7.170.62:42379 was elected leader, sending a full tablet report...
I20260929 02:23:12.106160 26574 ts_tablet_manager.cc:1428] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260929 02:23:12.105993 26572 consensus_queue.cc:260] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.107168 26419 heartbeater.cc:503] Master 127.7.170.62:42379 was elected leader, sending a full tablet report...
I20260929 02:23:12.110751 26572 ts_tablet_manager.cc:1428] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Time spent starting tablet: real 0.017s user 0.008s sys 0.006s
I20260929 02:23:12.110944 26573 raft_consensus.cc:348] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.111340 26573 raft_consensus.cc:374] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:12.111438 26573 raft_consensus.cc:729] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efed5199d67474595568931335aad9d, State: Initialized, Role: FOLLOWER
I20260929 02:23:12.112051 26573 consensus_queue.cc:260] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.113620 26573 ts_tablet_manager.cc:1428] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
W20260929 02:23:12.122731 26420 tablet.cc:2369] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:12.181521 26580 raft_consensus.cc:482] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:12.181869 26580 raft_consensus.cc:504] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.184010 26580 leader_election.cc:288] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0e4f4c401bb4c22853cdec1f9d6a2f8 (127.7.170.2:41841), 095f7ed737a848d0b9b4e9189ff2772c (127.7.170.3:37049)
I20260929 02:23:12.191612 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.184551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:50077 (local address 127.7.170.2:41841)
0929 02:23:12.184829 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.184837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.185491 (+ 654us) server_negotiation.cc:407] Connection header received
0929 02:23:12.185805 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.185813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.185906 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.185976 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:12.186740 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.187620 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.189531 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.190539 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.190668 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.190776 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.191119 (+ 343us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.191256 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260929 02:23:12.191720 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.185816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:53355 (local address 127.7.170.3:37049)
0929 02:23:12.186047 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.186061 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.186295 (+ 234us) server_negotiation.cc:407] Connection header received
0929 02:23:12.186495 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.186501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.186571 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.186648 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:12.187293 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.188035 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.189833 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.190754 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.190874 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.190983 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.191324 (+ 341us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.191455 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260929 02:23:12.191725 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.184466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:41841 (local address 127.7.170.1:50077)
0929 02:23:12.185301 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:12.185321 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:12.185533 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:12.186187 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:12.186191 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:12.186217 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:12.186601 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.186611 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.187748 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.187752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.189377 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.189390 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.190751 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.190757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.190909 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.190937 (+ 28us) client_negotiation.cc:769] Sending connection context
0929 02:23:12.191070 (+ 133us) client_negotiation.cc:240] Negotiation successful
0929 02:23:12.191208 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":44,"thread_start_us":475,"threads_started":1}
I20260929 02:23:12.191861 26582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.185462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:37049 (local address 127.7.170.1:53355)
0929 02:23:12.186172 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:12.186187 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:12.186349 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:12.186689 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:12.186693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:12.186772 (+ 79us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:12.187140 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.187151 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.188177 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.188182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.189697 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:12.189711 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.190978 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.190984 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:12.191167 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.191201 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:23:12.191271 (+ 70us) client_negotiation.cc:240] Negotiation successful
0929 02:23:12.191393 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":1,"thread_start_us":335,"threads_started":1}
I20260929 02:23:12.193269 26374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835414268ec94006ada69450d382bb30" candidate_uuid: "8efed5199d67474595568931335aad9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" is_pre_election: true
I20260929 02:23:12.193269 26508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835414268ec94006ada69450d382bb30" candidate_uuid: "8efed5199d67474595568931335aad9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095f7ed737a848d0b9b4e9189ff2772c" is_pre_election: true
I20260929 02:23:12.193940 26508 raft_consensus.cc:2454] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8efed5199d67474595568931335aad9d in term 0.
I20260929 02:23:12.193949 26374 raft_consensus.cc:2454] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8efed5199d67474595568931335aad9d in term 0.
I20260929 02:23:12.194672 26173 leader_election.cc:302] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8efed5199d67474595568931335aad9d, e0e4f4c401bb4c22853cdec1f9d6a2f8; no voters:
I20260929 02:23:12.195070 26580 raft_consensus.cc:2781] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:12.195238 26580 raft_consensus.cc:482] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:12.195369 26580 raft_consensus.cc:3037] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 0 FOLLOWER]: Advancing to term 1
W20260929 02:23:12.198058 26554 tablet.cc:2369] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:12.197980 26580 raft_consensus.cc:504] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.198565 26580 leader_election.cc:288] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [CANDIDATE]: Term 1 election: Requested vote from peers e0e4f4c401bb4c22853cdec1f9d6a2f8 (127.7.170.2:41841), 095f7ed737a848d0b9b4e9189ff2772c (127.7.170.3:37049)
I20260929 02:23:12.199186 26508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835414268ec94006ada69450d382bb30" candidate_uuid: "8efed5199d67474595568931335aad9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095f7ed737a848d0b9b4e9189ff2772c"
I20260929 02:23:12.199224 26374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "835414268ec94006ada69450d382bb30" candidate_uuid: "8efed5199d67474595568931335aad9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8"
I20260929 02:23:12.199421 26508 raft_consensus.cc:3037] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:12.199421 26374 raft_consensus.cc:3037] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:12.202802 26374 raft_consensus.cc:2454] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8efed5199d67474595568931335aad9d in term 1.
I20260929 02:23:12.202793 26508 raft_consensus.cc:2454] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8efed5199d67474595568931335aad9d in term 1.
I20260929 02:23:12.203197 26173 leader_election.cc:302] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8efed5199d67474595568931335aad9d, e0e4f4c401bb4c22853cdec1f9d6a2f8; no voters:
I20260929 02:23:12.203459 26580 raft_consensus.cc:2781] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:12.203933 26580 raft_consensus.cc:686] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 1 LEADER]: Becoming Leader. State: Replica: 8efed5199d67474595568931335aad9d, State: Running, Role: LEADER
I20260929 02:23:12.204458 26580 consensus_queue.cc:237] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } }
I20260929 02:23:12.209749 26095 catalog_manager.cc:5495] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 8efed5199d67474595568931335aad9d (127.7.170.1). New cstate: current_term: 1 leader_uuid: "8efed5199d67474595568931335aad9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8efed5199d67474595568931335aad9d" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 39915 } health_report { overall_health: HEALTHY } } }
I20260929 02:23:12.261186 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.254671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.7.170.1:39915)
0929 02:23:12.254908 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.254913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.254935 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:23:12.255097 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.255101 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.255198 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.255353 (+ 155us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:12.255802 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.256544 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.258584 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.258997 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.259106 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.259789 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:12.260443 (+ 654us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:12.260595 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.260790 (+ 195us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.260850 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 02:23:12.271202 26374 raft_consensus.cc:1261] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 1 FOLLOWER]: Refusing update from remote peer 8efed5199d67474595568931335aad9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:12.271201 26508 raft_consensus.cc:1261] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 1 FOLLOWER]: Refusing update from remote peer 8efed5199d67474595568931335aad9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:12.272181 26583 consensus_queue.cc:1035] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0e4f4c401bb4c22853cdec1f9d6a2f8" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:12.272553 26580 consensus_queue.cc:1035] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "095f7ed737a848d0b9b4e9189ff2772c" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 37049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:12.302837 26096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:12.303133 26096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:12.306804 26096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 57514b2345a34d3788072bcf8ca4ca24: Sending DeleteTablet for 3 replicas of tablet 835414268ec94006ada69450d382bb30
I20260929 02:23:12.308073 26488 tablet_service.cc:1514] Processing DeleteTablet for tablet 835414268ec94006ada69450d382bb30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:12 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:43540
I20260929 02:23:12.308207 26220 tablet_service.cc:1514] Processing DeleteTablet for tablet 835414268ec94006ada69450d382bb30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:12 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:36822
I20260929 02:23:12.308290 26354 tablet_service.cc:1514] Processing DeleteTablet for tablet 835414268ec94006ada69450d382bb30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:12 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:43290
I20260929 02:23:12.309024 26597 tablet_replica.cc:330] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: stopping tablet replica
I20260929 02:23:12.309535 26597 raft_consensus.cc:2229] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:12.310134 26597 raft_consensus.cc:2258] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:12.312286 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26157
I20260929 02:23:12.312786 26597 ts_tablet_manager.cc:1905] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:12.320372 26597 ts_tablet_manager.cc:1918] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:12.320675 26597 log.cc:1196] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/835414268ec94006ada69450d382bb30
I20260929 02:23:12.320793 26598 tablet_replica.cc:330] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: stopping tablet replica
I20260929 02:23:12.321329 26598 raft_consensus.cc:2229] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:12.321411 26597 ts_tablet_manager.cc:1939] T 835414268ec94006ada69450d382bb30 P 095f7ed737a848d0b9b4e9189ff2772c: Deleting consensus metadata
I20260929 02:23:12.322073 26598 raft_consensus.cc:2258] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:12.322872 26599 tablet_replica.cc:330] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: stopping tablet replica
I20260929 02:23:12.323333 26599 raft_consensus.cc:2229] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:12.323679 26080 catalog_manager.cc:4841] TS 095f7ed737a848d0b9b4e9189ff2772c (127.7.170.3:37049): tablet 835414268ec94006ada69450d382bb30 (table test-table [id=e364e41b786e41f5ba88eafa397f9e33]) successfully deleted
I20260929 02:23:12.323900 26599 raft_consensus.cc:2258] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:12.326030 26599 ts_tablet_manager.cc:1905] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:12.326036 26598 ts_tablet_manager.cc:1905] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:12.332952 26599 ts_tablet_manager.cc:1918] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:12.333132 26599 log.cc:1196] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/835414268ec94006ada69450d382bb30
I20260929 02:23:12.333213 26598 ts_tablet_manager.cc:1918] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:12.333385 26598 log.cc:1196] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/835414268ec94006ada69450d382bb30
I20260929 02:23:12.333676 26599 ts_tablet_manager.cc:1939] T 835414268ec94006ada69450d382bb30 P e0e4f4c401bb4c22853cdec1f9d6a2f8: Deleting consensus metadata
I20260929 02:23:12.333845 26598 ts_tablet_manager.cc:1939] T 835414268ec94006ada69450d382bb30 P 8efed5199d67474595568931335aad9d: Deleting consensus metadata
Sep 29 02:23:12 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:23:12.335578 26082 catalog_manager.cc:4841] TS 8efed5199d67474595568931335aad9d (127.7.170.1:39915): tablet 835414268ec94006ada69450d382bb30 (table test-table [id=e364e41b786e41f5ba88eafa397f9e33]) successfully deleted
I20260929 02:23:12.335595 26080 catalog_manager.cc:4841] TS e0e4f4c401bb4c22853cdec1f9d6a2f8 (127.7.170.2:41841): tablet 835414268ec94006ada69450d382bb30 (table test-table [id=e364e41b786e41f5ba88eafa397f9e33]) successfully deleted
I20260929 02:23:12.342932 26569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.313806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36842 (local address 127.7.170.1:39915)
0929 02:23:12.313998 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.314006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.324252 (+ 10246us) server_negotiation.cc:407] Connection header received
0929 02:23:12.324534 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.324539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.324648 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.324733 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:12.326751 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.328055 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.330029 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.330746 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.330859 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.336901 (+ 6042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:12.336913 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:12.336950 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:12.336999 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:12.340029 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.340835 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.340842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.340844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.341068 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.341437 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.341441 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.341443 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.341798 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:12.342034 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.342434 (+ 400us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.342618 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:12.343639 26280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:12.456895 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26157
I20260929 02:23:12.472754 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26291
Sep 29 02:23:12 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:23:12.488431 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.473540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.7.170.2:41841)
0929 02:23:12.473728 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.473734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.473754 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:23:12.473926 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.473930 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.473997 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.474084 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:12.475397 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.476155 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.477478 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.477857 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.477967 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.482228 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:12.482243 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:12.482261 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:12.482304 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:12.485196 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.486170 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.486179 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.486181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.486366 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.486810 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.486815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.486819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.487240 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:12.487521 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.487818 (+ 297us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.488051 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:23:12.489132 26414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:12.577629 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26291
I20260929 02:23:12.592025 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26425
Sep 29 02:23:12 dist-test-slave-8cs2 krb5kdc[26051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:23:12.607268 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:12.592749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.7.170.3:37049)
0929 02:23:12.592960 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:12.592966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:12.592988 (+ 22us) server_negotiation.cc:407] Connection header received
0929 02:23:12.593139 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:12.593142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:12.593211 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:12.593293 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:12.594655 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.595388 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.596746 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:12.597169 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:12.597258 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:12.601582 (+ 4324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:12.601599 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:12.601616 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:12.601655 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:12.604435 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.605214 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.605222 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.605224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.605361 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:12.605757 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:12.605762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:12.605764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:12.606151 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:12.606368 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:12.606730 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 02:23:12.606933 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:23:12.607918 26548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:12.697571 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26425
I20260929 02:23:12.712145 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26064
I20260929 02:23:12.713268 26126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:12.828387 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26064
2026-09-29T02:23:12Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4263 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-29T02:23:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:12Z Disabled control of system clock
I20260929 02:23:12.905453 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39489
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:39489
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:13.256758 26608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:13.257135 26608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:13.257226 26608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:13.264276 26608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:13.264376 26608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:13.264413 26608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:13.264469 26608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:13.264511 26608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:13.273483 26608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:39489
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:39489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:13.274727 26608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:13.276489 26608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:13.287457 26613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:13.287490 26616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:13.287523 26614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:13.288278 26608 server_base.cc:1034] running on GCE node
I20260929 02:23:13.289034 26608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:13.290437 26608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:13.291636 26608 hybrid_clock.cc:648] HybridClock initialized: now 1790648593291585 us; error 50 us; skew 500 ppm
I20260929 02:23:13.291963 26608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:13.294130 26608 webserver.cc:460] Webserver started at http://127.7.170.62:45755/ using document root <none> and password file <none>
I20260929 02:23:13.294795 26608 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:13.294883 26608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:13.295187 26608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:13.297994 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "42006c1a8d514496be288fff53dd557d"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "f403440991cced08030c135ce97e3c86"
server_key_iv: "12ae377c13399c195c80132d0b5ef50e"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.298758 26608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "42006c1a8d514496be288fff53dd557d"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "f403440991cced08030c135ce97e3c86"
server_key_iv: "12ae377c13399c195c80132d0b5ef50e"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.303696 26608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260929 02:23:13.306932 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:13.308190 26608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:13.308346 26608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "42006c1a8d514496be288fff53dd557d"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "f403440991cced08030c135ce97e3c86"
server_key_iv: "12ae377c13399c195c80132d0b5ef50e"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.308542 26608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:13.325332 26608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:13.326138 26608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:13.326386 26608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:13.343609 26608 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:39489
I20260929 02:23:13.343631 26673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:39489 every 8 connection(s)
I20260929 02:23:13.344907 26608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:13.348627 26674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:13.350818 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26608
I20260929 02:23:13.351011 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:13.351524 7848 external_mini_cluster.cc:1428] Setting key de296e23bbe6c72229263976c35416ac
I20260929 02:23:13.360486 26674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: Bootstrap starting.
I20260929 02:23:13.363950 26674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:13.364800 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:13.353444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49706 (local address 127.7.170.62:39489)
0929 02:23:13.354387 (+ 943us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:13.354397 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:13.354438 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:23:13.356040 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:13.356063 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:13.356527 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:13.356853 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:13.357435 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.358740 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.360534 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.361283 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.361426 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:13.362599 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:13.362607 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:13.362620 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:13.363119 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:13.363210 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:13.363229 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:13.363315 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:13.363513 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 02:23:13.363804 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":321,"threads_started":1}
I20260929 02:23:13.365028 26674 log.cc:824] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:13.367892 26674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: No bootstrap required, opened a new log
I20260929 02:23:13.374960 26674 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } }
I20260929 02:23:13.375275 26674 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:13.375348 26674 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42006c1a8d514496be288fff53dd557d, State: Initialized, Role: FOLLOWER
I20260929 02:23:13.375856 26674 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } }
I20260929 02:23:13.376056 26674 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:13.376147 26674 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:13.376266 26674 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:13.378315 26674 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } }
I20260929 02:23:13.378751 26674 leader_election.cc:302] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42006c1a8d514496be288fff53dd557d; no voters:
I20260929 02:23:13.379235 26674 leader_election.cc:288] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:13.379367 26679 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:13.380016 26679 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [term 1 LEADER]: Becoming Leader. State: Replica: 42006c1a8d514496be288fff53dd557d, State: Running, Role: LEADER
I20260929 02:23:13.380510 26679 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } }
I20260929 02:23:13.381181 26674 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:13.386859 26681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42006c1a8d514496be288fff53dd557d. Latest consensus state: current_term: 1 leader_uuid: "42006c1a8d514496be288fff53dd557d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } } }
I20260929 02:23:13.386965 26680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42006c1a8d514496be288fff53dd557d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42006c1a8d514496be288fff53dd557d" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 39489 } } }
I20260929 02:23:13.387315 26681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:13.387321 26680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d [sys.catalog]: This master's current role is: LEADER
W20260929 02:23:13.390940 26693 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:23:13.391047 26693 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:23:13.391165 26694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:13.392704 26694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:13.400684 26694 catalog_manager.cc:1347] Generated new cluster ID: 98e9175f68a44905aeb307cca2ad5085
I20260929 02:23:13.400792 26694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:13.409422 26694 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:13.411191 26694 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:13.418331 26694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: Generated new TSK 0
I20260929 02:23:13.419327 26694 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:13.430841 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--builtin_ntp_servers=127.7.170.20:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:13.745798 26698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:13.746060 26698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:13.746166 26698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:13.753101 26698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:13.753186 26698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:13.753328 26698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:13.761915 26698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:13.763118 26698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:13.764781 26698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:13.776651 26704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:13.776681 26706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:13.777087 26703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:13.777236 26698 server_base.cc:1034] running on GCE node
I20260929 02:23:13.777802 26698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:13.778723 26698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:13.779918 26698 hybrid_clock.cc:648] HybridClock initialized: now 1790648593779869 us; error 37 us; skew 500 ppm
I20260929 02:23:13.780200 26698 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:13.782294 26698 webserver.cc:460] Webserver started at http://127.7.170.1:33819/ using document root <none> and password file <none>
I20260929 02:23:13.782991 26698 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:13.783087 26698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:13.783389 26698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:13.786119 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "cd07fa5e58ce4848aa9a519e4f235f2c"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "c23bbc11ef3927928ebae65faf1a97ca"
server_key_iv: "a99525ed9aabdc36d27affbe6ab4642f"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.786832 26698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "cd07fa5e58ce4848aa9a519e4f235f2c"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "c23bbc11ef3927928ebae65faf1a97ca"
server_key_iv: "a99525ed9aabdc36d27affbe6ab4642f"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.791533 26698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260929 02:23:13.794704 26712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:13.795953 26698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260929 02:23:13.796098 26698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "cd07fa5e58ce4848aa9a519e4f235f2c"
format_stamp: "Formatted at 2026-09-29 02:23:13 on dist-test-slave-8cs2"
server_key: "c23bbc11ef3927928ebae65faf1a97ca"
server_key_iv: "a99525ed9aabdc36d27affbe6ab4642f"
server_key_version: "encryptionkey@0"
I20260929 02:23:13.796284 26698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:13.810356 26698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:13.811158 26698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:13.811400 26698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:13.812391 26698 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:13.813855 26698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:13.813951 26698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:13.814055 26698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:13.814121 26698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:13.842233 26698 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:38661
I20260929 02:23:13.842278 26824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:38661 every 8 connection(s)
I20260929 02:23:13.843454 26698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:13.844554 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26698
I20260929 02:23:13.844732 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:13.845304 7848 external_mini_cluster.cc:1428] Setting key e811963bc5130db8a490cc758530bde0
I20260929 02:23:13.848171 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--builtin_ntp_servers=127.7.170.20:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:13.855446 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:13.846224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:41859 (local address 127.7.170.62:39489)
0929 02:23:13.846432 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:13.846438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:13.847820 (+ 1382us) server_negotiation.cc:407] Connection header received
0929 02:23:13.849615 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:13.849620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:13.849694 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:13.849777 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:13.850806 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.851498 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.852994 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.853401 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.853509 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:13.854727 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:13.854731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:13.854734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:13.854754 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:13.854782 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:13.854793 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:13.854874 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:13.855061 (+ 187us) server_negotiation.cc:299] Negotiation successful
0929 02:23:13.855152 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:23:13.856434 26827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:13.846740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39489 (local address 127.7.170.1:41859)
0929 02:23:13.847684 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:13.847719 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:13.849373 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:13.850011 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:13.850019 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:13.850107 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:13.850652 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:13.850668 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.851637 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.851641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:13.852850 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:13.852863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:13.853602 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:13.853610 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:13.853818 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:13.854440 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:13.854442 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:13.854480 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:13.854551 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:13.854875 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:13.854886 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:13.854894 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:23:13.855040 (+ 146us) client_negotiation.cc:240] Negotiation successful
0929 02:23:13.855417 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":329,"threads_started":1}
I20260929 02:23:13.857811 26825 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39489
I20260929 02:23:13.858108 26825 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:13.858982 26825 heartbeater.cc:511] Master 127.7.170.62:39489 requested a full tablet report, sending...
I20260929 02:23:13.860963 26639 ts_manager.cc:194] Registered new tserver with Master: cd07fa5e58ce4848aa9a519e4f235f2c (127.7.170.1:38661)
I20260929 02:23:13.862407 26639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.1:41859
W20260929 02:23:14.161280 26828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:14.161535 26828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:14.161654 26828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:14.168605 26828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:14.168684 26828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:14.168823 26828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:14.177399 26828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:14.178531 26828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:14.180157 26828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:14.192535 26833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:14.192566 26836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:14.192615 26834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:14.193529 26828 server_base.cc:1034] running on GCE node
I20260929 02:23:14.194083 26828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:14.194919 26828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:14.196108 26828 hybrid_clock.cc:648] HybridClock initialized: now 1790648594196061 us; error 43 us; skew 500 ppm
I20260929 02:23:14.196390 26828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:14.198462 26828 webserver.cc:460] Webserver started at http://127.7.170.2:34731/ using document root <none> and password file <none>
I20260929 02:23:14.199095 26828 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:14.199205 26828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:14.199496 26828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:14.202242 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "2b6566959e114f55b81e53afdf01bee9"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "64364f5fe5a60acad3dd070d8d59e75f"
server_key_iv: "a6642f36f03b3571c90380ad49074045"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.202981 26828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "2b6566959e114f55b81e53afdf01bee9"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "64364f5fe5a60acad3dd070d8d59e75f"
server_key_iv: "a6642f36f03b3571c90380ad49074045"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.207693 26828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260929 02:23:14.211043 26842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.212447 26828 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:14.212598 26828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "2b6566959e114f55b81e53afdf01bee9"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "64364f5fe5a60acad3dd070d8d59e75f"
server_key_iv: "a6642f36f03b3571c90380ad49074045"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.212798 26828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:14.231014 26828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:14.231788 26828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:14.232045 26828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:14.233028 26828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:14.234445 26828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:14.234537 26828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.234642 26828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:14.234716 26828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.263466 26828 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:44915
I20260929 02:23:14.263504 26954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:44915 every 8 connection(s)
I20260929 02:23:14.264693 26828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:14.273595 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26828
I20260929 02:23:14.273765 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:14.274196 7848 external_mini_cluster.cc:1428] Setting key 4e1c6575cf8c20e0f9f72d27a773cd75
I20260929 02:23:14.276289 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.267019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:60397 (local address 127.7.170.62:39489)
0929 02:23:14.267231 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.267237 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.268575 (+ 1338us) server_negotiation.cc:407] Connection header received
0929 02:23:14.270269 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.270273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.270347 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.270431 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:14.271383 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.272110 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.273749 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.274158 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.274263 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.275558 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:14.275563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:14.275566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:14.275582 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:14.275612 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:14.275623 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:14.275696 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.275884 (+ 188us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.275973 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260929 02:23:14.276911 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--builtin_ntp_servers=127.7.170.20:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:14.277295 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.267584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39489 (local address 127.7.170.2:60397)
0929 02:23:14.268432 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.268471 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.270038 (+ 1567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.270599 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.270607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.270688 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:14.271239 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.271254 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.272226 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.272229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.273590 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.273606 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.274372 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.274380 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.274649 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.275281 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:14.275283 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:14.275318 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:14.275391 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:14.275706 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:14.275716 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:14.275724 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.275865 (+ 141us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.276233 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":253,"threads_started":1}
I20260929 02:23:14.278477 26955 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39489
I20260929 02:23:14.278779 26955 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:14.279623 26955 heartbeater.cc:511] Master 127.7.170.62:39489 requested a full tablet report, sending...
I20260929 02:23:14.281163 26639 ts_manager.cc:194] Registered new tserver with Master: 2b6566959e114f55b81e53afdf01bee9 (127.7.170.2:44915)
I20260929 02:23:14.282136 26639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.2:60397
W20260929 02:23:14.591419 26958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:14.591677 26958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:14.591785 26958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:14.598933 26958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:14.599027 26958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:14.599188 26958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:14.608090 26958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:39489
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:14.609328 26958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:14.611128 26958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:14.623557 26964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:14.623536 26963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:14.623605 26966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:14.624202 26958 server_base.cc:1034] running on GCE node
I20260929 02:23:14.624814 26958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:14.625747 26958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:14.626955 26958 hybrid_clock.cc:648] HybridClock initialized: now 1790648594626904 us; error 44 us; skew 500 ppm
I20260929 02:23:14.627241 26958 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:14.629350 26958 webserver.cc:460] Webserver started at http://127.7.170.3:38619/ using document root <none> and password file <none>
I20260929 02:23:14.630013 26958 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:14.630111 26958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:14.630391 26958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:14.633184 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "f5dcd645fe87460c827e2205d68e4c8b"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "1c20368522dba7434f0062ae113fb9ea"
server_key_iv: "90f0a03254406f377925ac6941349cd1"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.633925 26958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "f5dcd645fe87460c827e2205d68e4c8b"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "1c20368522dba7434f0062ae113fb9ea"
server_key_iv: "90f0a03254406f377925ac6941349cd1"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.638720 26958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260929 02:23:14.642073 26972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.643404 26958 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:23:14.643554 26958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "f5dcd645fe87460c827e2205d68e4c8b"
format_stamp: "Formatted at 2026-09-29 02:23:14 on dist-test-slave-8cs2"
server_key: "1c20368522dba7434f0062ae113fb9ea"
server_key_iv: "90f0a03254406f377925ac6941349cd1"
server_key_version: "encryptionkey@0"
I20260929 02:23:14.643746 26958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:14.667277 26958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:14.668072 26958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:14.668321 26958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:14.669270 26958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:14.670718 26958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:14.670801 26958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.670898 26958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:14.670972 26958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:14.700973 26958 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:46149
I20260929 02:23:14.701048 27084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:46149 every 8 connection(s)
I20260929 02:23:14.702242 26958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:14.702891 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 26958
I20260929 02:23:14.703061 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:14.703615 7848 external_mini_cluster.cc:1428] Setting key 360a1caf08f18d69652a48843b1593c0
I20260929 02:23:14.714148 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.704710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:41451 (local address 127.7.170.62:39489)
0929 02:23:14.704898 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.704904 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.706554 (+ 1650us) server_negotiation.cc:407] Connection header received
0929 02:23:14.708299 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.708303 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.708368 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.708461 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:14.709519 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.710215 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.711808 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.712210 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.712301 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.713532 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:14.713536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:14.713540 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:14.713559 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:14.713591 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:14.713603 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:14.713670 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.713833 (+ 163us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.713909 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260929 02:23:14.715124 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.705332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:39489 (local address 127.7.170.3:41451)
0929 02:23:14.706385 (+ 1053us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.706441 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.708033 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.708612 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.708619 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.708723 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:14.709344 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.709361 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.710347 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.710352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.711687 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.711700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.712385 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.712390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.712599 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.713242 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:14.713244 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:14.713284 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:14.713363 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:14.713677 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:14.713682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:14.713687 (+ 5us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.713809 (+ 122us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.714108 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":277,"threads_started":1}
I20260929 02:23:14.716491 27085 heartbeater.cc:347] Connected to a master server at 127.7.170.62:39489
I20260929 02:23:14.716770 27085 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:14.717639 27085 heartbeater.cc:511] Master 127.7.170.62:39489 requested a full tablet report, sending...
I20260929 02:23:14.719035 26639 ts_manager.cc:194] Registered new tserver with Master: f5dcd645fe87460c827e2205d68e4c8b (127.7.170.3:46149)
I20260929 02:23:14.719771 26639 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.3:41451
I20260929 02:23:14.730829 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:14.741710 26677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.735843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49716 (local address 127.7.170.62:39489)
0929 02:23:14.736090 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.736096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.736407 (+ 311us) server_negotiation.cc:407] Connection header received
0929 02:23:14.736616 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.736622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.736754 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.736844 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:14.737451 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.738332 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.739668 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.740053 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.740137 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.741093 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:14.741097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:14.741101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:14.741117 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:14.741148 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:14.741159 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:14.741262 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.741422 (+ 160us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.741470 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260929 02:23:14.745322 26639 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:14.751960 26639 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:14.782900 27100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.763466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44915 (local address 127.0.0.1:33124)
0929 02:23:14.764496 (+ 1030us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.764517 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.764615 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.776116 (+ 11501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.776125 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.776154 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:14.776523 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.776535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.778724 (+ 2189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.778729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.780454 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.780468 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.782156 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.782163 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.782311 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.782352 (+ 41us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.782427 (+ 75us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.782595 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":921,"mutex_wait_us":1,"thread_start_us":11035,"threads_started":1}
I20260929 02:23:14.783990 27097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.762847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:38661 (local address 127.0.0.1:35992)
0929 02:23:14.763826 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.763876 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.764045 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.777019 (+ 12974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.777023 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.777048 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:14.777329 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.777338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.779097 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.779102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.781438 (+ 2336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.781460 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.783225 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.783231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.783391 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.783422 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.783493 (+ 71us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.783627 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":38,"thread_start_us":398,"threads_started":1}
I20260929 02:23:14.784096 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.764144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33124 (local address 127.7.170.2:44915)
0929 02:23:14.775427 (+ 11283us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.775437 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.775468 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:23:14.775583 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.775592 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.775862 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.776726 (+ 864us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:14.777008 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.778585 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.780606 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.781962 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.783365 (+ 1403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.783596 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.783659 (+ 63us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.783788 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11100,"thread_start_us":11703,"threads_started":1}
I20260929 02:23:14.784253 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.764025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:46149 (local address 127.0.0.1:44296)
0929 02:23:14.774781 (+ 10756us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.774806 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.774927 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.776537 (+ 1610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.776541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.776568 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:14.776909 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.776920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.779011 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.779016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.781408 (+ 2392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.781423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.783515 (+ 2092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.783521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.783697 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.783727 (+ 30us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.783796 (+ 69us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.783914 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10626,"mutex_wait_us":94,"thread_start_us":688,"threads_started":1}
I20260929 02:23:14.784476 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.764560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44296 (local address 127.7.170.3:46149)
0929 02:23:14.775938 (+ 11378us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.775947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.775988 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:23:14.776092 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.776099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.776320 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.776523 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:14.777164 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.778888 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.781645 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.783015 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.783144 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.783419 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.783885 (+ 466us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.784094 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11225,"thread_start_us":11087,"threads_started":1}
I20260929 02:23:14.785226 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.763035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.7.170.1:38661)
0929 02:23:14.776362 (+ 13327us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.776378 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.776404 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:23:14.776542 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.776550 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.776822 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.777037 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:14.777472 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.778970 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.781655 (+ 2685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.783031 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.784555 (+ 1524us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.784759 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.784807 (+ 48us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.784919 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13182,"thread_start_us":13092,"threads_started":1}
I20260929 02:23:14.788062 26890 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (DEFAULT_TABLE table=test-table [id=17d12c00082d45d69cc38b36b4f808ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:14.788357 27020 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (DEFAULT_TABLE table=test-table [id=17d12c00082d45d69cc38b36b4f808ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:14.789510 26890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c17f7934a430fb8c7d69401e6dbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:14.789649 27020 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c17f7934a430fb8c7d69401e6dbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:14.790488 26760 tablet_service.cc:1467] Processing CreateTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (DEFAULT_TABLE table=test-table [id=17d12c00082d45d69cc38b36b4f808ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:14.791643 26760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e3c17f7934a430fb8c7d69401e6dbd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:14.802107 27103 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Bootstrap starting.
I20260929 02:23:14.803400 27104 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Bootstrap starting.
I20260929 02:23:14.806869 27103 tablet_bootstrap.cc:654] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:14.806870 27104 tablet_bootstrap.cc:654] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:14.807684 27105 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Bootstrap starting.
I20260929 02:23:14.808205 27103 log.cc:824] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:14.808207 27104 log.cc:824] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:14.810729 27105 tablet_bootstrap.cc:654] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:14.811444 27104 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: No bootstrap required, opened a new log
I20260929 02:23:14.811681 27104 ts_tablet_manager.cc:1397] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260929 02:23:14.811905 27105 log.cc:824] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:14.813797 27103 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: No bootstrap required, opened a new log
I20260929 02:23:14.814069 27103 ts_tablet_manager.cc:1397] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260929 02:23:14.818470 27105 tablet_bootstrap.cc:492] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: No bootstrap required, opened a new log
I20260929 02:23:14.818760 27105 ts_tablet_manager.cc:1397] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20260929 02:23:14.822032 27104 raft_consensus.cc:348] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.822465 27104 raft_consensus.cc:374] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:14.822598 27104 raft_consensus.cc:729] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5dcd645fe87460c827e2205d68e4c8b, State: Initialized, Role: FOLLOWER
I20260929 02:23:14.823338 27104 consensus_queue.cc:260] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.824425 27103 raft_consensus.cc:348] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.824872 27103 raft_consensus.cc:374] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:14.825033 27103 raft_consensus.cc:729] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd07fa5e58ce4848aa9a519e4f235f2c, State: Initialized, Role: FOLLOWER
I20260929 02:23:14.825503 27085 heartbeater.cc:503] Master 127.7.170.62:39489 was elected leader, sending a full tablet report...
I20260929 02:23:14.825688 27103 consensus_queue.cc:260] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.826205 27105 raft_consensus.cc:348] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.826640 27105 raft_consensus.cc:374] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:14.826725 27105 raft_consensus.cc:729] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b6566959e114f55b81e53afdf01bee9, State: Initialized, Role: FOLLOWER
I20260929 02:23:14.826865 26825 heartbeater.cc:503] Master 127.7.170.62:39489 was elected leader, sending a full tablet report...
I20260929 02:23:14.827200 27105 consensus_queue.cc:260] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.827401 27103 ts_tablet_manager.cc:1428] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20260929 02:23:14.828256 26955 heartbeater.cc:503] Master 127.7.170.62:39489 was elected leader, sending a full tablet report...
I20260929 02:23:14.828676 27105 ts_tablet_manager.cc:1428] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260929 02:23:14.829736 27104 ts_tablet_manager.cc:1428] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Time spent starting tablet: real 0.018s user 0.015s sys 0.000s
W20260929 02:23:14.845551 26826 tablet.cc:2369] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:14.916322 27110 raft_consensus.cc:482] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:14.916730 27110 raft_consensus.cc:504] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.918509 27110 leader_election.cc:288] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5dcd645fe87460c827e2205d68e4c8b (127.7.170.3:46149), 2b6566959e114f55b81e53afdf01bee9 (127.7.170.2:44915)
I20260929 02:23:14.924842 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.918904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:59259 (local address 127.7.170.3:46149)
0929 02:23:14.919100 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.919106 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.919861 (+ 755us) server_negotiation.cc:407] Connection header received
0929 02:23:14.920049 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.920054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.920114 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.920188 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:14.920719 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.921463 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.923246 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.924006 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.924114 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.924209 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.924494 (+ 285us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.924608 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:23:14.924868 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.919028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:45499 (local address 127.7.170.2:44915)
0929 02:23:14.919299 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.919305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.919972 (+ 667us) server_negotiation.cc:407] Connection header received
0929 02:23:14.920135 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.920139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.920198 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.920271 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:14.920871 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.921586 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.923286 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.924022 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.924130 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.924215 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.924503 (+ 288us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.924630 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120}
I20260929 02:23:14.924980 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.918947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:46149 (local address 127.7.170.1:59259)
0929 02:23:14.919755 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.919770 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.919910 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.920216 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.920220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.920242 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:14.920585 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.920594 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.921588 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.921592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.923072 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.923083 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.924191 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.924198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.924335 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.924366 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.924452 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.924578 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":45,"thread_start_us":414,"threads_started":1}
I20260929 02:23:14.925024 27113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.919028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:44915 (local address 127.7.170.1:45499)
0929 02:23:14.919837 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:14.919885 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:14.920010 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:14.920302 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:14.920307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:14.920344 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:14.920717 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.920726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.921720 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.921728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.923120 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:14.923132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.924208 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.924214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:14.924364 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.924386 (+ 22us) client_negotiation.cc:769] Sending connection context
0929 02:23:14.924459 (+ 73us) client_negotiation.cc:240] Negotiation successful
0929 02:23:14.924578 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"mutex_wait_us":8,"thread_start_us":453,"threads_started":1}
I20260929 02:23:14.926000 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c17f7934a430fb8c7d69401e6dbd8" candidate_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5dcd645fe87460c827e2205d68e4c8b" is_pre_election: true
I20260929 02:23:14.926029 26910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c17f7934a430fb8c7d69401e6dbd8" candidate_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6566959e114f55b81e53afdf01bee9" is_pre_election: true
I20260929 02:23:14.926559 26910 raft_consensus.cc:2454] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd07fa5e58ce4848aa9a519e4f235f2c in term 0.
I20260929 02:23:14.926560 27040 raft_consensus.cc:2454] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd07fa5e58ce4848aa9a519e4f235f2c in term 0.
I20260929 02:23:14.927175 26714 leader_election.cc:302] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b6566959e114f55b81e53afdf01bee9, cd07fa5e58ce4848aa9a519e4f235f2c; no voters:
I20260929 02:23:14.927537 27110 raft_consensus.cc:2781] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:14.927691 27110 raft_consensus.cc:482] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:14.927794 27110 raft_consensus.cc:3037] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:14.929898 27110 raft_consensus.cc:504] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.930423 27110 leader_election.cc:288] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [CANDIDATE]: Term 1 election: Requested vote from peers f5dcd645fe87460c827e2205d68e4c8b (127.7.170.3:46149), 2b6566959e114f55b81e53afdf01bee9 (127.7.170.2:44915)
I20260929 02:23:14.930780 27040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c17f7934a430fb8c7d69401e6dbd8" candidate_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5dcd645fe87460c827e2205d68e4c8b"
I20260929 02:23:14.930868 26910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e3c17f7934a430fb8c7d69401e6dbd8" candidate_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b6566959e114f55b81e53afdf01bee9"
I20260929 02:23:14.930949 27040 raft_consensus.cc:3037] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:14.931051 26910 raft_consensus.cc:3037] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:14.933135 27040 raft_consensus.cc:2454] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd07fa5e58ce4848aa9a519e4f235f2c in term 1.
I20260929 02:23:14.933328 26910 raft_consensus.cc:2454] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd07fa5e58ce4848aa9a519e4f235f2c in term 1.
I20260929 02:23:14.933496 26713 leader_election.cc:302] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd07fa5e58ce4848aa9a519e4f235f2c, f5dcd645fe87460c827e2205d68e4c8b; no voters:
I20260929 02:23:14.933765 27110 raft_consensus.cc:2781] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:14.934213 27110 raft_consensus.cc:686] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 1 LEADER]: Becoming Leader. State: Replica: cd07fa5e58ce4848aa9a519e4f235f2c, State: Running, Role: LEADER
I20260929 02:23:14.934660 27110 consensus_queue.cc:237] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } }
I20260929 02:23:14.938791 26639 catalog_manager.cc:5495] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c reported cstate change: term changed from 0 to 1, leader changed from <none> to cd07fa5e58ce4848aa9a519e4f235f2c (127.7.170.1). New cstate: current_term: 1 leader_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd07fa5e58ce4848aa9a519e4f235f2c" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 38661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 } health_report { overall_health: UNKNOWN } } }
W20260929 02:23:14.953966 27086 tablet.cc:2369] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:14.975653 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:14.970447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.7.170.1:38661)
0929 02:23:14.970651 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:14.970655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:14.970674 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:23:14.970827 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:14.970831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:14.970918 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:14.971050 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:14.971353 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.972052 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.973483 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:14.973926 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:14.974043 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:14.974586 (+ 543us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:14.975083 (+ 497us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:14.975238 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:14.975379 (+ 141us) server_negotiation.cc:299] Negotiation successful
0929 02:23:14.975422 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:23:14.983709 26910 raft_consensus.cc:1261] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 1 FOLLOWER]: Refusing update from remote peer cd07fa5e58ce4848aa9a519e4f235f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:14.983716 27040 raft_consensus.cc:1261] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 1 FOLLOWER]: Refusing update from remote peer cd07fa5e58ce4848aa9a519e4f235f2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:14.984602 27114 consensus_queue.cc:1035] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b6566959e114f55b81e53afdf01bee9" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 44915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:14.984951 27110 consensus_queue.cc:1035] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5dcd645fe87460c827e2205d68e4c8b" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 46149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:14.996356 27117 mvcc.cc:205] Tried to move back new op lower bound from 7334496645043605504 to 7334496644855767040. Current Snapshot: MvccSnapshot[applied={T|T < 7334496645043605504}]
I20260929 02:23:14.997910 27119 mvcc.cc:205] Tried to move back new op lower bound from 7334496645043605504 to 7334496644855767040. Current Snapshot: MvccSnapshot[applied={T|T < 7334496645043605504}]
I20260929 02:23:15.009421 26639 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:15.009648 26639 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:15.012980 26639 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 42006c1a8d514496be288fff53dd557d: Sending DeleteTablet for 3 replicas of tablet 2e3c17f7934a430fb8c7d69401e6dbd8
I20260929 02:23:15.014297 26890 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:15 UTC) from {username='slave'} at 127.0.0.1:33124
I20260929 02:23:15.014541 26760 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:15 UTC) from {username='slave'} at 127.0.0.1:35992
I20260929 02:23:15.015244 27020 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e3c17f7934a430fb8c7d69401e6dbd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:15 UTC) from {username='slave'} at 127.0.0.1:44296
I20260929 02:23:15.015424 27127 tablet_replica.cc:330] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: stopping tablet replica
I20260929 02:23:15.015902 27127 raft_consensus.cc:2229] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:15.016208 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26698
I20260929 02:23:15.016650 27127 raft_consensus.cc:2258] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:15.018538 27127 ts_tablet_manager.cc:1905] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:15.025867 27127 ts_tablet_manager.cc:1918] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:15.026117 27127 log.cc:1196] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/2e3c17f7934a430fb8c7d69401e6dbd8
I20260929 02:23:15.026738 27127 ts_tablet_manager.cc:1939] T 2e3c17f7934a430fb8c7d69401e6dbd8 P cd07fa5e58ce4848aa9a519e4f235f2c: Deleting consensus metadata
I20260929 02:23:15.028587 26623 catalog_manager.cc:4841] TS cd07fa5e58ce4848aa9a519e4f235f2c (127.7.170.1:38661): tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (table test-table [id=17d12c00082d45d69cc38b36b4f808ab]) successfully deleted
I20260929 02:23:15.032240 27126 tablet_replica.cc:330] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: stopping tablet replica
I20260929 02:23:15.032907 27126 raft_consensus.cc:2229] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:15.033573 27126 raft_consensus.cc:2258] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:15.034587 27128 tablet_replica.cc:330] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: stopping tablet replica
I20260929 02:23:15.034796 27098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:15.022819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35996 (local address 127.7.170.1:38661)
0929 02:23:15.023052 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:15.023058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:15.023408 (+ 350us) server_negotiation.cc:407] Connection header received
0929 02:23:15.024227 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:15.024233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:15.024330 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:15.024425 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:15.025539 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.026900 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.030804 (+ 3904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.031475 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.031605 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:15.033081 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:15.033093 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:15.033110 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:15.033898 (+ 788us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:15.033965 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:15.033991 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:15.034081 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:15.034315 (+ 234us) server_negotiation.cc:299] Negotiation successful
0929 02:23:15.034423 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 02:23:15.035871 27126 ts_tablet_manager.cc:1905] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:15.035933 26820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:15.035570 27128 raft_consensus.cc:2229] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:15.036906 27128 raft_consensus.cc:2258] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:15.038892 27128 ts_tablet_manager.cc:1905] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:15.042639 27126 ts_tablet_manager.cc:1918] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:15.042779 27126 log.cc:1196] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/2e3c17f7934a430fb8c7d69401e6dbd8
I20260929 02:23:15.043182 27126 ts_tablet_manager.cc:1939] T 2e3c17f7934a430fb8c7d69401e6dbd8 P 2b6566959e114f55b81e53afdf01bee9: Deleting consensus metadata
I20260929 02:23:15.044615 26624 catalog_manager.cc:4841] TS 2b6566959e114f55b81e53afdf01bee9 (127.7.170.2:44915): tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (table test-table [id=17d12c00082d45d69cc38b36b4f808ab]) successfully deleted
I20260929 02:23:15.045843 27128 ts_tablet_manager.cc:1918] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:15.046003 27128 log.cc:1196] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/2e3c17f7934a430fb8c7d69401e6dbd8
I20260929 02:23:15.046480 27128 ts_tablet_manager.cc:1939] T 2e3c17f7934a430fb8c7d69401e6dbd8 P f5dcd645fe87460c827e2205d68e4c8b: Deleting consensus metadata
I20260929 02:23:15.048211 26623 catalog_manager.cc:4841] TS f5dcd645fe87460c827e2205d68e4c8b (127.7.170.3:46149): tablet 2e3c17f7934a430fb8c7d69401e6dbd8 (table test-table [id=17d12c00082d45d69cc38b36b4f808ab]) successfully deleted
I20260929 02:23:15.131840 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26698
I20260929 02:23:15.146854 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26828
I20260929 02:23:15.153007 27101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:15.147628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.7.170.2:44915)
0929 02:23:15.147785 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:15.147791 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:15.147811 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:23:15.147901 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:15.147905 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:15.147980 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:15.148058 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:15.148413 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.149238 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.150389 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.150785 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.150882 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:15.151733 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:15.151741 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:15.151764 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:15.152331 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:15.152378 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:15.152398 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:15.152479 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:15.152573 (+ 94us) server_negotiation.cc:299] Negotiation successful
0929 02:23:15.152656 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260929 02:23:15.153620 26950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:15.239014 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26828
I20260929 02:23:15.252767 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26958
I20260929 02:23:15.259040 27102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:15.253498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44306 (local address 127.7.170.3:46149)
0929 02:23:15.253670 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:15.253675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:15.253694 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:23:15.253814 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:15.253818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:15.253901 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:15.253999 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:15.254371 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.255112 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.256309 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:15.256718 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:15.256821 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:15.257798 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:15.257806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:15.257821 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:15.258366 (+ 545us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:15.258410 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:15.258430 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:15.258526 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:15.258649 (+ 123us) server_negotiation.cc:299] Negotiation successful
0929 02:23:15.258751 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260929 02:23:15.259603 27080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:15.345631 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26958
I20260929 02:23:15.366118 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 26608
I20260929 02:23:15.367089 26669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:15.472069 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 26608
2026-09-29T02:23:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2640 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:15 dist-test-slave-8cs2 krb5kdc[27136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:15 dist-test-slave-8cs2 krb5kdc[27136](info): set up 2 sockets
Sep 29 02:23:15 dist-test-slave-8cs2 krb5kdc[27136](info): commencing operation
krb5kdc: starting...
W20260929 02:23:16.521299 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.012s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:16 dist-test-slave-8cs2 krb5kdc[27136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:16.778054 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34277
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:34277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:17.096592 27149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:17.096909 27149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:17.097055 27149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:17.104399 27149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:17.104513 27149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:17.104581 27149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:17.104658 27149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:17.104717 27149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:17.114781 27149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:34277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:17.116088 27149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:17.117645 27149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:17.128785 27154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:17.128723 27155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:17.128763 27157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:17.129751 27149 server_base.cc:1034] running on GCE node
I20260929 02:23:17.130491 27149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:17.131785 27149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:17.133024 27149 hybrid_clock.cc:648] HybridClock initialized: now 1790648597132965 us; error 44 us; skew 500 ppm
I20260929 02:23:17.133339 27149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:17 dist-test-slave-8cs2 krb5kdc[27136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:17.142073 27149 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:17.143538 27149 webserver.cc:460] Webserver started at http://127.7.170.62:41375/ using document root <none> and password file <none>
I20260929 02:23:17.144255 27149 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:17.144441 27149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:17.144752 27149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:17.147810 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "de6312da57224047834b0b252b0e18f6"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "e38312e73bf936bb31bb08ecab1c8d3d"
server_key_iv: "62e5b7163312ac5dae7ef50e92350239"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.148540 27149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "de6312da57224047834b0b252b0e18f6"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "e38312e73bf936bb31bb08ecab1c8d3d"
server_key_iv: "62e5b7163312ac5dae7ef50e92350239"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.153493 27149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:23:17.156683 27164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:17.157938 27149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:17.158092 27149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "de6312da57224047834b0b252b0e18f6"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "e38312e73bf936bb31bb08ecab1c8d3d"
server_key_iv: "62e5b7163312ac5dae7ef50e92350239"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.158255 27149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:17.169466 27149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:17.173352 27149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:17.173585 27149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:17.190222 27149 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:34277
I20260929 02:23:17.190249 27215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:34277 every 8 connection(s)
I20260929 02:23:17.191561 27149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:17.195202 27216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:17.201350 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27149
I20260929 02:23:17.201534 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:17.202080 7848 external_mini_cluster.cc:1428] Setting key c9a938cd11d31c911b9122c68136a717
I20260929 02:23:17.206059 27216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: Bootstrap starting.
I20260929 02:23:17.209854 27216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:17.210934 27216 log.cc:824] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:17.214228 27216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: No bootstrap required, opened a new log
Sep 29 02:23:17 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:17.223469 27216 raft_consensus.cc:348] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } }
I20260929 02:23:17.223968 27216 raft_consensus.cc:374] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:17.224087 27216 raft_consensus.cc:729] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de6312da57224047834b0b252b0e18f6, State: Initialized, Role: FOLLOWER
I20260929 02:23:17.224774 27216 consensus_queue.cc:260] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } }
I20260929 02:23:17.225061 27216 raft_consensus.cc:388] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:17.225219 27216 raft_consensus.cc:482] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:17.225404 27216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:17.225951 27219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:17.203990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.7.170.62:34277)
0929 02:23:17.204778 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:17.204790 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:17.204831 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:23:17.206593 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:17.206617 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:17.207008 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:17.207343 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:17.209361 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.210808 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.212318 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.213094 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.213236 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:17.218179 (+ 4943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:17.218187 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:17.218203 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:17.218245 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:17.221415 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:17.222285 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:17.222291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:17.222302 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:17.222471 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:17.222885 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:17.222888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:17.222890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:17.223413 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:17.223634 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:17.224248 (+ 614us) server_negotiation.cc:299] Negotiation successful
0929 02:23:17.224665 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":342,"threads_started":1}
I20260929 02:23:17.227772 27216 raft_consensus.cc:504] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } }
I20260929 02:23:17.228260 27216 leader_election.cc:302] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de6312da57224047834b0b252b0e18f6; no voters:
I20260929 02:23:17.228888 27216 leader_election.cc:288] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:17.229000 27221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:17.229606 27221 raft_consensus.cc:686] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [term 1 LEADER]: Becoming Leader. State: Replica: de6312da57224047834b0b252b0e18f6, State: Running, Role: LEADER
I20260929 02:23:17.230108 27221 consensus_queue.cc:237] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } }
I20260929 02:23:17.230809 27216 sys_catalog.cc:564] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:17.232777 27222 sys_catalog.cc:455] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de6312da57224047834b0b252b0e18f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } } }
I20260929 02:23:17.233117 27223 sys_catalog.cc:455] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de6312da57224047834b0b252b0e18f6. Latest consensus state: current_term: 1 leader_uuid: "de6312da57224047834b0b252b0e18f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de6312da57224047834b0b252b0e18f6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34277 } } }
I20260929 02:23:17.233325 27222 sys_catalog.cc:458] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:17.233388 27223 sys_catalog.cc:458] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:17.234620 27229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:17.239513 27229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:17.248852 27229 catalog_manager.cc:1347] Generated new cluster ID: 237685056b284697bf7f3ad24cfda289
I20260929 02:23:17.248992 27229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:17.264998 27229 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:17.266307 27229 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:17.278301 27229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: Generated new TSK 0
I20260929 02:23:17.279148 27229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.170.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
Entry for principal kudu/127.7.170.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab.
I20260929 02:23:17.344599 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:17.663674 27242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:17.663939 27242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:17.664073 27242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:17.671309 27242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:17.671427 27242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:17.671593 27242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:17.681447 27242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.1.keytab
--principal=kudu/127.7.170.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:17.682736 27242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:17.684068 27242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:17.697873 27247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:17.697826 27248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:17.697795 27250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:17.698448 27242 server_base.cc:1034] running on GCE node
I20260929 02:23:17.699999 27242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:17.701089 27242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:17.702307 27242 hybrid_clock.cc:648] HybridClock initialized: now 1790648597702249 us; error 53 us; skew 500 ppm
I20260929 02:23:17.702628 27242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:17 dist-test-slave-8cs2 krb5kdc[27136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:17.705780 27242 init.cc:377] Logged in from keytab as kudu/127.7.170.1@KRBTEST.COM (short username kudu)
I20260929 02:23:17.707259 27242 webserver.cc:460] Webserver started at http://127.7.170.1:36833/ using document root <none> and password file <none>
I20260929 02:23:17.707978 27242 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:17.708072 27242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:17.708329 27242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:17.711262 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "f2f398b5743441b0be70fa88a6ff88e7"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "6e5b18ef7461ef40147282a7bade0595"
server_key_iv: "c22c4639047016b8f0ca2cc36cee0e51"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.712004 27242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "f2f398b5743441b0be70fa88a6ff88e7"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "6e5b18ef7461ef40147282a7bade0595"
server_key_iv: "c22c4639047016b8f0ca2cc36cee0e51"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.717159 27242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260929 02:23:17.720620 27257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:17.721982 27242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:17.722122 27242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "f2f398b5743441b0be70fa88a6ff88e7"
format_stamp: "Formatted at 2026-09-29 02:23:17 on dist-test-slave-8cs2"
server_key: "6e5b18ef7461ef40147282a7bade0595"
server_key_iv: "c22c4639047016b8f0ca2cc36cee0e51"
server_key_version: "encryptionkey@0"
I20260929 02:23:17.722287 27242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:17.747601 27242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:17.751436 27242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:17.751669 27242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:17.752630 27242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:17.753955 27242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:17.754026 27242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:17.754112 27242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:17.754153 27242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:17.783213 27242 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:46561
I20260929 02:23:17.783264 27369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:46561 every 8 connection(s)
I20260929 02:23:17.784584 27242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:17.789192 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27242
I20260929 02:23:17.789353 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:17.789769 7848 external_mini_cluster.cc:1428] Setting key 447132c55e4bc56a3e58a88d90f42fbf
Sep 29 02:23:17 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648597, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.1@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:17.806072 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:17.787390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:57059 (local address 127.7.170.62:34277)
0929 02:23:17.787889 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:17.787893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:17.788845 (+ 952us) server_negotiation.cc:407] Connection header received
0929 02:23:17.791061 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:17.791066 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:17.791145 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:17.791248 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:17.793376 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.794096 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.796099 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.796497 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.796588 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:17.800326 (+ 3738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:17.800332 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:17.800335 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:17.800368 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:17.803131 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:17.803817 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:17.803823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:17.803826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:17.803965 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:17.804316 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:17.804320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:17.804323 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:17.804650 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:17.804790 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:17.805490 (+ 700us) server_negotiation.cc:299] Negotiation successful
0929 02:23:17.805702 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260929 02:23:17.806854 27373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:17.787893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34277 (local address 127.7.170.1:57059)
0929 02:23:17.788660 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:17.788706 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:17.790801 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:17.791499 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:17.791511 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:17.792251 (+ 740us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:17.793197 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:17.793221 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.794250 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.794255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:17.795961 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:17.795979 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:17.796703 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:17.796710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:17.796991 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:17.797641 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:17.797668 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:17.800118 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:17.803268 (+ 3150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:17.803275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:17.803289 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:17.803696 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:17.804051 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:17.804054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:17.804056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:17.804219 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:17.804765 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:17.804771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:17.805126 (+ 355us) client_negotiation.cc:769] Sending connection context
0929 02:23:17.805345 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:23:17.805739 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":263,"threads_started":1}
I20260929 02:23:17.808298 27370 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34277
I20260929 02:23:17.808627 27370 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:17.809487 27370 heartbeater.cc:511] Master 127.7.170.62:34277 requested a full tablet report, sending...
I20260929 02:23:17.811594 27181 ts_manager.cc:194] Registered new tserver with Master: f2f398b5743441b0be70fa88a6ff88e7 (127.7.170.1:46561)
I20260929 02:23:17.813069 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.1@KRBTEST.COM'} at 127.7.170.1:57059
WARNING: no policy specified for kudu/127.7.170.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
Entry for principal kudu/127.7.170.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab.
I20260929 02:23:17.848766 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:18.167672 27376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:18.167961 27376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:18.168098 27376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:18.175293 27376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:18.175398 27376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:18.175541 27376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:18.184815 27376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.2.keytab
--principal=kudu/127.7.170.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:18.186093 27376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:18.187383 27376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:18.200971 27382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:18.201090 27381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:18.201463 27384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:18.202006 27376 server_base.cc:1034] running on GCE node
I20260929 02:23:18.202652 27376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:18.203711 27376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:18.204970 27376 hybrid_clock.cc:648] HybridClock initialized: now 1790648598204891 us; error 53 us; skew 500 ppm
I20260929 02:23:18.205307 27376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:18 dist-test-slave-8cs2 krb5kdc[27136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:18.208899 27376 init.cc:377] Logged in from keytab as kudu/127.7.170.2@KRBTEST.COM (short username kudu)
I20260929 02:23:18.210266 27376 webserver.cc:460] Webserver started at http://127.7.170.2:33431/ using document root <none> and password file <none>
I20260929 02:23:18.210952 27376 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:18.211061 27376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:18.211333 27376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:18.214166 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "256409a9c76847029e88418739d75f89"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "2b071ab8e2a9501a6756bbf905ba2ffd"
server_key_iv: "67c6249b985e7dde24bb08e5765f4718"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.214895 27376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "256409a9c76847029e88418739d75f89"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "2b071ab8e2a9501a6756bbf905ba2ffd"
server_key_iv: "67c6249b985e7dde24bb08e5765f4718"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.219723 27376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 02:23:18.223022 27391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.224298 27376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:18.224462 27376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "256409a9c76847029e88418739d75f89"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "2b071ab8e2a9501a6756bbf905ba2ffd"
server_key_iv: "67c6249b985e7dde24bb08e5765f4718"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.224725 27376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:18.244468 27376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:18.248776 27376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:18.249130 27376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:18.250047 27376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:18.251375 27376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:18.251470 27376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.251595 27376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:18.251648 27376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.280701 27376 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:41051
I20260929 02:23:18.280743 27503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:41051 every 8 connection(s)
I20260929 02:23:18.281966 27376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:18.282353 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27376
I20260929 02:23:18.282506 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:18.283053 7848 external_mini_cluster.cc:1428] Setting key 012d3092c8837a304d7c91d32f9005d7
Sep 29 02:23:18 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.2@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:18.302444 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.284628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:34607 (local address 127.7.170.62:34277)
0929 02:23:18.284818 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.284824 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.286165 (+ 1341us) server_negotiation.cc:407] Connection header received
0929 02:23:18.287950 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.287956 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.288033 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.288114 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:18.289687 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.290389 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.291936 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.292356 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.292476 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.296347 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:18.296354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:18.296359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:18.296398 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:18.299405 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.300087 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.300093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.300096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.300244 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.300652 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.300656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.300659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.301020 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:18.301154 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.301878 (+ 724us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.302114 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:23:18.303246 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.285202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34277 (local address 127.7.170.2:34607)
0929 02:23:18.285988 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.286031 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.287689 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.288370 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.288378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.288901 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:18.289522 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.289540 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.290536 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.290539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.291795 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.291808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.292617 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.292624 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.292891 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.293583 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:18.293615 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:18.296094 (+ 2479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:18.299542 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:18.299549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:18.299561 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:18.299956 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:18.300341 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:18.300345 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:18.300347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:18.300545 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:18.301158 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:18.301164 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:18.301499 (+ 335us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.301729 (+ 230us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.302130 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":306,"threads_started":1}
I20260929 02:23:18.304715 27504 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34277
I20260929 02:23:18.305060 27504 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:18.305778 27504 heartbeater.cc:511] Master 127.7.170.62:34277 requested a full tablet report, sending...
I20260929 02:23:18.307325 27181 ts_manager.cc:194] Registered new tserver with Master: 256409a9c76847029e88418739d75f89 (127.7.170.2:41051)
I20260929 02:23:18.308133 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.2@KRBTEST.COM'} at 127.7.170.2:34607
WARNING: no policy specified for kudu/127.7.170.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
Entry for principal kudu/127.7.170.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab.
I20260929 02:23:18.340672 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:36049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:18.664494 27509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:18.664764 27509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:18.664880 27509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:18.672324 27509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:18.672435 27509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:18.672587 27509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:18.682585 27509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:36049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.3.keytab
--principal=kudu/127.7.170.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:18.683928 27509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:18.685245 27509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:18.698478 27514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:18.698503 27515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:18.699007 27517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:18.699314 27509 server_base.cc:1034] running on GCE node
I20260929 02:23:18.699888 27509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:18.700814 27509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:18.702023 27509 hybrid_clock.cc:648] HybridClock initialized: now 1790648598701966 us; error 44 us; skew 500 ppm
I20260929 02:23:18.702343 27509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:18 dist-test-slave-8cs2 krb5kdc[27136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:18.705387 27509 init.cc:377] Logged in from keytab as kudu/127.7.170.3@KRBTEST.COM (short username kudu)
I20260929 02:23:18.706802 27509 webserver.cc:460] Webserver started at http://127.7.170.3:34265/ using document root <none> and password file <none>
I20260929 02:23:18.707496 27509 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:18.707612 27509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:18.707896 27509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:18.710803 27509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "a240c44af7984147ab2563fa02cb2891"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "e025311cfa23fbc0ca491c85cb9f74de"
server_key_iv: "a0cac9ee8380b16428bacb095cda97b4"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.711572 27509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "a240c44af7984147ab2563fa02cb2891"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "e025311cfa23fbc0ca491c85cb9f74de"
server_key_iv: "a0cac9ee8380b16428bacb095cda97b4"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.716595 27509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:23:18.720196 27524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.721575 27509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260929 02:23:18.721724 27509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "a240c44af7984147ab2563fa02cb2891"
format_stamp: "Formatted at 2026-09-29 02:23:18 on dist-test-slave-8cs2"
server_key: "e025311cfa23fbc0ca491c85cb9f74de"
server_key_iv: "a0cac9ee8380b16428bacb095cda97b4"
server_key_version: "encryptionkey@0"
I20260929 02:23:18.721921 27509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:18.736793 27509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:18.741632 27509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:18.741894 27509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:18.742794 27509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:18.744091 27509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:18.744163 27509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.744275 27509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:18.744326 27509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:18.771574 27509 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:34315
I20260929 02:23:18.771620 27636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:34315 every 8 connection(s)
I20260929 02:23:18.772921 27509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:18.775146 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27509
I20260929 02:23:18.775319 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:18.775851 7848 external_mini_cluster.cc:1428] Setting key ca0f1b36d009d1eae06336afe1b55ef4
Sep 29 02:23:18 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648598, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.3@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:18.793310 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.775509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:53019 (local address 127.7.170.62:34277)
0929 02:23:18.775673 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.775677 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.777227 (+ 1550us) server_negotiation.cc:407] Connection header received
0929 02:23:18.779459 (+ 2232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.779463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.779531 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.779607 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:18.781149 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.781908 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.783555 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.784178 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.784305 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.787842 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:18.787848 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:18.787851 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:18.787882 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:18.790466 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.791313 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.791317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.791320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.791443 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.791829 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.791832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.791834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.792093 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:18.792201 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.792868 (+ 667us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.793068 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:23:18.794349 27639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.776172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34277 (local address 127.7.170.3:53019)
0929 02:23:18.777072 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.777128 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.779212 (+ 2084us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.779802 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.779810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.780369 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:18.780981 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.781001 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.782041 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.782045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.783400 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.783413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.784399 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.784405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.784599 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.785315 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:18.785339 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:18.787621 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:18.790600 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:18.790609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:18.790627 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:18.791161 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:18.791555 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:18.791558 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:18.791561 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:18.791746 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:18.792228 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:18.792234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:18.792562 (+ 328us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.792790 (+ 228us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.793238 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":327,"threads_started":1}
I20260929 02:23:18.795658 27637 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34277
I20260929 02:23:18.795960 27637 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:18.796656 27637 heartbeater.cc:511] Master 127.7.170.62:34277 requested a full tablet report, sending...
I20260929 02:23:18.798060 27181 ts_manager.cc:194] Registered new tserver with Master: a240c44af7984147ab2563fa02cb2891 (127.7.170.3:34315)
I20260929 02:23:18.798832 27181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.170.3@KRBTEST.COM'} at 127.7.170.3:53019
I20260929 02:23:18.803398 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:18.816474 27370 heartbeater.cc:503] Master 127.7.170.62:34277 was elected leader, sending a full tablet report...
I20260929 02:23:18.819669 27372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.807321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44114 (local address 127.7.170.62:34277)
0929 02:23:18.807535 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.807541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.807780 (+ 239us) server_negotiation.cc:407] Connection header received
0929 02:23:18.807965 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.807969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.808030 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.808116 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:18.809519 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.810261 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.811535 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.811970 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.812062 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.814303 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:18.814308 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:18.814312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:18.814345 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:18.816975 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.817665 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.817670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.817674 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.817850 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:18.818231 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:18.818237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:18.818241 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:18.818584 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:18.818820 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.819142 (+ 322us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.819368 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:23:18.823597 27181 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:18.831079 27181 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:18.865528 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.843571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.7.170.2:41051)
0929 02:23:18.855928 (+ 12357us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.855937 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.855969 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:23:18.856077 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.856085 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.856389 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.858876 (+ 2487us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:18.859098 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.860533 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.862987 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864318 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.864449 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.864656 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.865038 (+ 382us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.865211 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12220,"thread_start_us":12124,"threads_started":1}
I20260929 02:23:18.865943 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.843498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50960 (local address 127.7.170.1:46561)
0929 02:23:18.856365 (+ 12867us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.856373 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.856399 (+ 26us) server_negotiation.cc:407] Connection header received
0929 02:23:18.856491 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.856499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.856740 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.856977 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:18.857882 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.859413 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.862835 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864318 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.864449 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.864695 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.865492 (+ 797us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.865656 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12690,"thread_start_us":13041,"threads_started":1}
I20260929 02:23:18.866443 27650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.843438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:41051 (local address 127.0.0.1:46142)
0929 02:23:18.854679 (+ 11241us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.854702 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.854856 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.856854 (+ 1998us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.856862 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.856896 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:18.857376 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.857393 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.860681 (+ 3288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.860687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.862814 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.862825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864503 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.864514 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.864711 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.864782 (+ 71us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.864911 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.865149 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11083,"mutex_wait_us":41,"thread_start_us":10908,"threads_started":1}
I20260929 02:23:18.866472 27653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.843205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:50960)
0929 02:23:18.852278 (+ 9073us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.852318 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.852534 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.857209 (+ 4675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.857215 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.857237 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:18.857672 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.857684 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.859579 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.859585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.862653 (+ 3068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.862667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864803 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.864808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.865028 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.865065 (+ 37us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.865143 (+ 78us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.865277 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8804,"mutex_wait_us":27,"thread_start_us":18472,"threads_started":1}
I20260929 02:23:18.870247 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.845170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:34315 (local address 127.0.0.1:45682)
0929 02:23:18.861964 (+ 16794us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.861983 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.862179 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.863052 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.863056 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.863080 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:18.863394 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.863404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864831 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.864835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.867741 (+ 2906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.867757 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.869489 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.869496 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.869654 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.869685 (+ 31us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.869771 (+ 86us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.869909 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16648,"mutex_wait_us":1,"thread_start_us":6710,"threads_started":1}
I20260929 02:23:18.870504 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.845819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45682 (local address 127.7.170.3:34315)
0929 02:23:18.861466 (+ 15647us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.861476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.862081 (+ 605us) server_negotiation.cc:407] Connection header received
0929 02:23:18.862354 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.862362 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.862568 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.862762 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:18.863550 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.864666 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.867906 (+ 3240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.869279 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.869423 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.869640 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.869846 (+ 206us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.869997 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15215,"thread_start_us":15385,"threads_started":1}
I20260929 02:23:18.870570 27439 tablet_service.cc:1467] Processing CreateTablet for tablet 799418b00b4347378697f9450bf8bf5e (DEFAULT_TABLE table=test-table [id=2310fe877be54ab4b8b0b2a77da5d2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:18.870570 27305 tablet_service.cc:1467] Processing CreateTablet for tablet 799418b00b4347378697f9450bf8bf5e (DEFAULT_TABLE table=test-table [id=2310fe877be54ab4b8b0b2a77da5d2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:18.871917 27305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799418b00b4347378697f9450bf8bf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:18.871948 27439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799418b00b4347378697f9450bf8bf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:18.874197 27572 tablet_service.cc:1467] Processing CreateTablet for tablet 799418b00b4347378697f9450bf8bf5e (DEFAULT_TABLE table=test-table [id=2310fe877be54ab4b8b0b2a77da5d2f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:18.875540 27572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 799418b00b4347378697f9450bf8bf5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:18.882439 27655 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Bootstrap starting.
I20260929 02:23:18.885627 27655 tablet_bootstrap.cc:654] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:18.886646 27655 log.cc:824] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:18.887634 27656 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Bootstrap starting.
I20260929 02:23:18.891855 27656 tablet_bootstrap.cc:654] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:18.893337 27656 log.cc:824] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:18.893765 27658 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Bootstrap starting.
I20260929 02:23:18.896354 27655 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: No bootstrap required, opened a new log
I20260929 02:23:18.896600 27656 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: No bootstrap required, opened a new log
I20260929 02:23:18.896654 27655 ts_tablet_manager.cc:1397] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.007s
I20260929 02:23:18.896847 27656 ts_tablet_manager.cc:1397] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260929 02:23:18.897701 27658 tablet_bootstrap.cc:654] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:18.898978 27658 log.cc:824] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:18.902179 27658 tablet_bootstrap.cc:492] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: No bootstrap required, opened a new log
I20260929 02:23:18.902462 27658 ts_tablet_manager.cc:1397] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260929 02:23:18.904235 27656 raft_consensus.cc:348] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.904692 27656 raft_consensus.cc:374] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:18.904788 27656 raft_consensus.cc:729] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 256409a9c76847029e88418739d75f89, State: Initialized, Role: FOLLOWER
I20260929 02:23:18.905545 27656 consensus_queue.cc:260] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.906646 27504 heartbeater.cc:503] Master 127.7.170.62:34277 was elected leader, sending a full tablet report...
I20260929 02:23:18.906675 27655 raft_consensus.cc:348] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.907153 27655 raft_consensus.cc:374] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:18.907246 27655 raft_consensus.cc:729] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f398b5743441b0be70fa88a6ff88e7, State: Initialized, Role: FOLLOWER
I20260929 02:23:18.907322 27656 ts_tablet_manager.cc:1428] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260929 02:23:18.907919 27655 consensus_queue.cc:260] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.909658 27655 ts_tablet_manager.cc:1428] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260929 02:23:18.913232 27658 raft_consensus.cc:348] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.913573 27658 raft_consensus.cc:374] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:18.913642 27658 raft_consensus.cc:729] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a240c44af7984147ab2563fa02cb2891, State: Initialized, Role: FOLLOWER
I20260929 02:23:18.914089 27658 consensus_queue.cc:260] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.915037 27637 heartbeater.cc:503] Master 127.7.170.62:34277 was elected leader, sending a full tablet report...
I20260929 02:23:18.915542 27658 ts_tablet_manager.cc:1428] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:23:18.943967 27661 raft_consensus.cc:482] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:18.944281 27661 raft_consensus.cc:504] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.946048 27661 leader_election.cc:288] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f398b5743441b0be70fa88a6ff88e7 (127.7.170.1:46561), a240c44af7984147ab2563fa02cb2891 (127.7.170.3:34315)
I20260929 02:23:18.955173 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.946482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:44345 (local address 127.7.170.1:46561)
0929 02:23:18.946677 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.946684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.947353 (+ 669us) server_negotiation.cc:407] Connection header received
0929 02:23:18.947548 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.947553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.947627 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.947719 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:18.948506 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.949678 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.952651 (+ 2973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.953949 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.954095 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.954203 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.954761 (+ 558us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.954906 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:23:18.955173 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.946512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:50197 (local address 127.7.170.3:34315)
0929 02:23:18.946677 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.946684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.947353 (+ 669us) server_negotiation.cc:407] Connection header received
0929 02:23:18.947548 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.947553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.947627 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.947719 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:18.948506 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.949691 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.952692 (+ 3001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.953970 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.954098 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.954202 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.954761 (+ 559us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.954933 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:18.955492 27665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.946376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:34315 (local address 127.7.170.2:50197)
0929 02:23:18.947246 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.947259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.947418 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.947739 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.947744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.947777 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:18.948333 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.948346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.949825 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.949829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.952450 (+ 2621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.952468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.954233 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.954241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.954526 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.954589 (+ 63us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.954690 (+ 101us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.954891 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":277,"spinlock_wait_cycles":25728,"thread_start_us":512,"threads_started":1}
I20260929 02:23:18.955695 27664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.946251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.7.170.2:44345)
0929 02:23:18.947229 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:18.947253 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:18.947418 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:18.947739 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:18.947744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:18.947777 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:18.948368 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.948380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.949893 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.949897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.952450 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:18.952468 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.954235 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.954240 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:18.954522 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.954590 (+ 68us) client_negotiation.cc:769] Sending connection context
0929 02:23:18.954698 (+ 108us) client_negotiation.cc:240] Negotiation successful
0929 02:23:18.954915 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":215,"spinlock_wait_cycles":17024,"thread_start_us":603,"threads_started":1}
I20260929 02:23:18.956570 27592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799418b00b4347378697f9450bf8bf5e" candidate_uuid: "256409a9c76847029e88418739d75f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a240c44af7984147ab2563fa02cb2891" is_pre_election: true
I20260929 02:23:18.956620 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799418b00b4347378697f9450bf8bf5e" candidate_uuid: "256409a9c76847029e88418739d75f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f398b5743441b0be70fa88a6ff88e7" is_pre_election: true
I20260929 02:23:18.957075 27592 raft_consensus.cc:2454] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256409a9c76847029e88418739d75f89 in term 0.
I20260929 02:23:18.957108 27325 raft_consensus.cc:2454] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 256409a9c76847029e88418739d75f89 in term 0.
I20260929 02:23:18.957736 27393 leader_election.cc:302] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256409a9c76847029e88418739d75f89, a240c44af7984147ab2563fa02cb2891; no voters:
I20260929 02:23:18.958119 27661 raft_consensus.cc:2781] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:18.958267 27661 raft_consensus.cc:482] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:18.958362 27661 raft_consensus.cc:3037] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:18.960510 27661 raft_consensus.cc:504] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.961153 27661 leader_election.cc:288] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [CANDIDATE]: Term 1 election: Requested vote from peers f2f398b5743441b0be70fa88a6ff88e7 (127.7.170.1:46561), a240c44af7984147ab2563fa02cb2891 (127.7.170.3:34315)
I20260929 02:23:18.961606 27325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799418b00b4347378697f9450bf8bf5e" candidate_uuid: "256409a9c76847029e88418739d75f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f398b5743441b0be70fa88a6ff88e7"
I20260929 02:23:18.961720 27592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "799418b00b4347378697f9450bf8bf5e" candidate_uuid: "256409a9c76847029e88418739d75f89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a240c44af7984147ab2563fa02cb2891"
I20260929 02:23:18.961787 27325 raft_consensus.cc:3037] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:18.961938 27592 raft_consensus.cc:3037] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:18.964318 27325 raft_consensus.cc:2454] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256409a9c76847029e88418739d75f89 in term 1.
I20260929 02:23:18.964491 27592 raft_consensus.cc:2454] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 256409a9c76847029e88418739d75f89 in term 1.
I20260929 02:23:18.964721 27394 leader_election.cc:302] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 256409a9c76847029e88418739d75f89, f2f398b5743441b0be70fa88a6ff88e7; no voters:
I20260929 02:23:18.965101 27661 raft_consensus.cc:2781] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:18.965533 27661 raft_consensus.cc:686] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 1 LEADER]: Becoming Leader. State: Replica: 256409a9c76847029e88418739d75f89, State: Running, Role: LEADER
I20260929 02:23:18.966035 27661 consensus_queue.cc:237] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } }
I20260929 02:23:18.970392 27181 catalog_manager.cc:5495] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 256409a9c76847029e88418739d75f89 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "256409a9c76847029e88418739d75f89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "256409a9c76847029e88418739d75f89" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 41051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 } health_report { overall_health: UNKNOWN } } }
I20260929 02:23:19.000202 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:18.994996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.7.170.2:41051)
0929 02:23:18.995233 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:18.995239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:18.995257 (+ 18us) server_negotiation.cc:407] Connection header received
0929 02:23:18.995397 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:18.995401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:18.995476 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:18.995599 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:18.995988 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.996730 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.998129 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:18.998539 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:18.998643 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:18.999107 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:18.999615 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:18.999761 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:18.999854 (+ 93us) server_negotiation.cc:299] Negotiation successful
0929 02:23:18.999935 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260929 02:23:19.008903 27592 raft_consensus.cc:1261] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 1 FOLLOWER]: Refusing update from remote peer 256409a9c76847029e88418739d75f89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:19.008903 27325 raft_consensus.cc:1261] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 1 FOLLOWER]: Refusing update from remote peer 256409a9c76847029e88418739d75f89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:19.010079 27661 consensus_queue.cc:1035] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f398b5743441b0be70fa88a6ff88e7" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 46561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:19.010393 27666 consensus_queue.cc:1035] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a240c44af7984147ab2563fa02cb2891" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 34315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:19.022233 27670 mvcc.cc:205] Tried to move back new op lower bound from 7334496661529092096 to 7334496661368655872. Current Snapshot: MvccSnapshot[applied={T|T < 7334496661529092096}]
I20260929 02:23:19.023739 27669 mvcc.cc:205] Tried to move back new op lower bound from 7334496661529092096 to 7334496661368655872. Current Snapshot: MvccSnapshot[applied={T|T < 7334496661529092096}]
I20260929 02:23:19.039435 27181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:19.039743 27181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:19.044327 27181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de6312da57224047834b0b252b0e18f6: Sending DeleteTablet for 3 replicas of tablet 799418b00b4347378697f9450bf8bf5e
I20260929 02:23:19.045475 27305 tablet_service.cc:1514] Processing DeleteTablet for tablet 799418b00b4347378697f9450bf8bf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:19 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:50960
I20260929 02:23:19.045905 27304 tablet_service.cc:1514] Processing DeleteTablet for tablet 799418b00b4347378697f9450bf8bf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:19 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:50960
I20260929 02:23:19.045995 27439 tablet_service.cc:1514] Processing DeleteTablet for tablet 799418b00b4347378697f9450bf8bf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:19 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:46142
I20260929 02:23:19.046720 27572 tablet_service.cc:1514] Processing DeleteTablet for tablet 799418b00b4347378697f9450bf8bf5e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:19 UTC) from {username='kudu', principal='kudu/127.7.170.62@KRBTEST.COM'} at 127.0.0.1:45682
I20260929 02:23:19.047719 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27242
I20260929 02:23:19.056685 27680 tablet_replica.cc:330] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: stopping tablet replica
I20260929 02:23:19.057227 27680 raft_consensus.cc:2229] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:19.057909 27680 raft_consensus.cc:2258] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:19.062196 27680 ts_tablet_manager.cc:1905] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:19.067821 27679 tablet_replica.cc:330] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: stopping tablet replica
I20260929 02:23:19.068259 27679 raft_consensus.cc:2229] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:19.068759 27679 raft_consensus.cc:2258] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7 [term 1 FOLLOWER]: Raft consensus is shut down!
Sep 29 02:23:19 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.1@KRBTEST.COM
I20260929 02:23:19.073436 27681 tablet_replica.cc:330] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: stopping tablet replica
I20260929 02:23:19.073813 27680 ts_tablet_manager.cc:1918] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:19.073872 27681 raft_consensus.cc:2229] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:19.073997 27680 log.cc:1196] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/799418b00b4347378697f9450bf8bf5e
I20260929 02:23:19.074415 27681 raft_consensus.cc:2258] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:19.074553 27680 ts_tablet_manager.cc:1939] T 799418b00b4347378697f9450bf8bf5e P 256409a9c76847029e88418739d75f89: Deleting consensus metadata
I20260929 02:23:19.076347 27165 catalog_manager.cc:4841] TS 256409a9c76847029e88418739d75f89 (127.7.170.2:41051): tablet 799418b00b4347378697f9450bf8bf5e (table test-table [id=2310fe877be54ab4b8b0b2a77da5d2f0]) successfully deleted
I20260929 02:23:19.076551 27681 ts_tablet_manager.cc:1905] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:19.078408 27679 ts_tablet_manager.cc:1905] T 799418b00b4347378697f9450bf8bf5e P f2f398b5743441b0be70fa88a6ff88e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:19.080081 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.049790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50968 (local address 127.7.170.1:46561)
0929 02:23:19.049973 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:19.049979 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:19.061291 (+ 11312us) server_negotiation.cc:407] Connection header received
0929 02:23:19.061496 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:19.061501 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:19.061607 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:19.061715 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:19.063633 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.064731 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.066741 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.067471 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.067568 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:19.073047 (+ 5479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:19.073062 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:19.073080 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:19.073121 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:19.076660 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.077526 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.077535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.077539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.077694 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.078154 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.078159 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.078162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.078622 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:19.078898 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:19.079468 (+ 570us) server_negotiation.cc:299] Negotiation successful
0929 02:23:19.079696 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260929 02:23:19.081422 27365 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:19.083489 27681 ts_tablet_manager.cc:1918] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:19.083669 27681 log.cc:1196] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/799418b00b4347378697f9450bf8bf5e
I20260929 02:23:19.084223 27681 ts_tablet_manager.cc:1939] T 799418b00b4347378697f9450bf8bf5e P a240c44af7984147ab2563fa02cb2891: Deleting consensus metadata
I20260929 02:23:19.086133 27166 catalog_manager.cc:4841] TS a240c44af7984147ab2563fa02cb2891 (127.7.170.3:34315): tablet 799418b00b4347378697f9450bf8bf5e (table test-table [id=2310fe877be54ab4b8b0b2a77da5d2f0]) successfully deleted
I20260929 02:23:19.173848 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27242
I20260929 02:23:19.188158 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27376
W20260929 02:23:19.188194 27167 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.170.1:46561 (error 108)
I20260929 02:23:19.190644 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.190108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:50970)
0929 02:23:19.190375 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.190479 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
W20260929 02:23:19.190917 27167 catalog_manager.cc:4584] TS f2f398b5743441b0be70fa88a6ff88e7 (127.7.170.1:46561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 799418b00b4347378697f9450bf8bf5e: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
I20260929 02:23:19.191886 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.191503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:50986)
0929 02:23:19.191676 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.191745 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
Sep 29 02:23:19 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.2@KRBTEST.COM
I20260929 02:23:19.207648 27652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.189178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46148 (local address 127.7.170.2:41051)
0929 02:23:19.189373 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:19.189379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:19.189400 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:23:19.189580 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:19.189586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:19.189683 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:19.189790 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:19.191696 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.192890 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.194843 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.195536 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.195659 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:19.200662 (+ 5003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:19.200680 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:19.200700 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:19.200745 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:19.204709 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.205523 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.205532 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.205536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.205703 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.206096 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.206101 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.206105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.206557 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:19.206881 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:19.207047 (+ 166us) server_negotiation.cc:299] Negotiation successful
0929 02:23:19.207271 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260929 02:23:19.208460 27499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:19.236413 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.235965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:50988)
0929 02:23:19.236160 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.236255 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 02:23:19.237421 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.237074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51004)
0929 02:23:19.237231 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.237299 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260929 02:23:19.244526 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.244161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51006)
0929 02:23:19.244321 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.244392 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260929 02:23:19.245455 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.245125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51016)
0929 02:23:19.245263 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.245338 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260929 02:23:19.286218 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.285608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51024)
0929 02:23:19.285953 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.286058 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217}
I20260929 02:23:19.287364 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.286961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51032)
0929 02:23:19.287126 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.287196 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 02:23:19.320866 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27376
I20260929 02:23:19.321909 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.321291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51040)
0929 02:23:19.321683 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.321777 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251}
I20260929 02:23:19.322847 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.322512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51042)
0929 02:23:19.322658 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.322724 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 02:23:19.337244 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27509
Sep 29 02:23:19 dist-test-slave-8cs2 krb5kdc[27136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.3@KRBTEST.COM
I20260929 02:23:19.351300 27654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.337900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.7.170.3:34315)
0929 02:23:19.338060 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:19.338065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:19.338109 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:23:19.338263 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:19.338266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:19.338353 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:19.338453 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:19.339776 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.340496 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.341786 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:19.342170 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:19.342259 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:19.346295 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:19.346309 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:19.346325 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:19.346364 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:19.348890 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.349478 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.349485 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.349488 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.349658 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:19.349993 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:19.349997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:19.349999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:19.350344 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:19.350610 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:19.350812 (+ 202us) server_negotiation.cc:299] Negotiation successful
0929 02:23:19.351000 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:19.351960 27632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:19.388327 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.387906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51048)
0929 02:23:19.388103 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.388188 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 02:23:19.389308 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.388917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51060)
0929 02:23:19.389108 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.389175 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260929 02:23:19.424655 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.424194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51068)
0929 02:23:19.424395 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.424515 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260929 02:23:19.425638 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.425278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51078)
0929 02:23:19.425416 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.425506 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260929 02:23:19.439838 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27509
I20260929 02:23:19.454123 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27149
I20260929 02:23:19.455259 27211 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:19.563145 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27149
I20260929 02:23:19.563180 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:19.562547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:46561 (local address 127.0.0.1:51088)
0929 02:23:19.562906 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:19.563015 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.1:46561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
2026-09-29T02:23:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4092 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-29T02:23:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:19Z Disabled control of system clock
I20260929 02:23:19.636621 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34841
--webserver_interface=127.7.170.62
--webserver_port=0
--builtin_ntp_servers=127.7.170.20:41377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:34841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:19.951346 27691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:19.951655 27691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:19.951746 27691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:19.958815 27691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:19.958925 27691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:19.958962 27691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:19.959017 27691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:19.959059 27691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:19.968031 27691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:34841
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:19.969272 27691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:19.970978 27691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:19.982122 27696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:19.982156 27699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:19.982189 27697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:19.983276 27691 server_base.cc:1034] running on GCE node
I20260929 02:23:19.984081 27691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:19.985491 27691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:19.986706 27691 hybrid_clock.cc:648] HybridClock initialized: now 1790648599986651 us; error 44 us; skew 500 ppm
I20260929 02:23:19.987037 27691 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:19.989267 27691 webserver.cc:460] Webserver started at http://127.7.170.62:46513/ using document root <none> and password file <none>
I20260929 02:23:19.989917 27691 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:19.990015 27691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:19.990309 27691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:19.993134 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "c9729b135dd741a0881f06b7414a0cc8"
format_stamp: "Formatted at 2026-09-29 02:23:19 on dist-test-slave-8cs2"
server_key: "57225af0e0bcb2e5f6c7bdd7242c0f72"
server_key_iv: "20e97e42c441536fa2dfed8fec2d640d"
server_key_version: "encryptionkey@0"
I20260929 02:23:19.993898 27691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "c9729b135dd741a0881f06b7414a0cc8"
format_stamp: "Formatted at 2026-09-29 02:23:19 on dist-test-slave-8cs2"
server_key: "57225af0e0bcb2e5f6c7bdd7242c0f72"
server_key_iv: "20e97e42c441536fa2dfed8fec2d640d"
server_key_version: "encryptionkey@0"
I20260929 02:23:19.998734 27691 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:20.001984 27705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.003211 27691 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:20.003368 27691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "c9729b135dd741a0881f06b7414a0cc8"
format_stamp: "Formatted at 2026-09-29 02:23:19 on dist-test-slave-8cs2"
server_key: "57225af0e0bcb2e5f6c7bdd7242c0f72"
server_key_iv: "20e97e42c441536fa2dfed8fec2d640d"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.003557 27691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:20.017866 27691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:20.018713 27691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:20.018985 27691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:20.035882 27691 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:34841
I20260929 02:23:20.035921 27756 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:34841 every 8 connection(s)
I20260929 02:23:20.037204 27691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:20.040038 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27691
I20260929 02:23:20.040210 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:20.040736 7848 external_mini_cluster.cc:1428] Setting key 7d0870daca9698cfdced97fd0e062558
I20260929 02:23:20.041708 27757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:20.056162 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:20.042667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34168 (local address 127.7.170.62:34841)
0929 02:23:20.043719 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:20.043734 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:20.043785 (+ 51us) server_negotiation.cc:407] Connection header received
0929 02:23:20.046259 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:20.046285 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:20.046895 (+ 610us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:20.047339 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:20.047877 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.049851 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.051252 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.052276 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.052467 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:20.053460 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:20.053469 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:20.053488 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:20.054194 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:20.054287 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:20.054317 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:20.054421 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:20.054622 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 02:23:20.054950 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":332,"threads_started":1}
I20260929 02:23:20.056156 27757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: Bootstrap starting.
I20260929 02:23:20.059283 27757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:20.060184 27757 log.cc:824] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:20.062726 27757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: No bootstrap required, opened a new log
I20260929 02:23:20.069828 27757 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } }
I20260929 02:23:20.070272 27757 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:20.070360 27757 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9729b135dd741a0881f06b7414a0cc8, State: Initialized, Role: FOLLOWER
I20260929 02:23:20.070866 27757 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } }
I20260929 02:23:20.071054 27757 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:20.071151 27757 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:20.071274 27757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:20.073599 27757 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } }
I20260929 02:23:20.074069 27757 leader_election.cc:302] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9729b135dd741a0881f06b7414a0cc8; no voters:
I20260929 02:23:20.074641 27757 leader_election.cc:288] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:20.074771 27762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:20.075498 27762 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [term 1 LEADER]: Becoming Leader. State: Replica: c9729b135dd741a0881f06b7414a0cc8, State: Running, Role: LEADER
I20260929 02:23:20.076014 27762 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } }
I20260929 02:23:20.076648 27757 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:20.078581 27764 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9729b135dd741a0881f06b7414a0cc8. Latest consensus state: current_term: 1 leader_uuid: "c9729b135dd741a0881f06b7414a0cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } } }
I20260929 02:23:20.078785 27763 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9729b135dd741a0881f06b7414a0cc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9729b135dd741a0881f06b7414a0cc8" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34841 } } }
I20260929 02:23:20.079056 27764 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:20.079059 27763 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:20.081269 27768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:20.086282 27768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:20.095744 27768 catalog_manager.cc:1347] Generated new cluster ID: d7b687baa5784201af586c630ab4cb25
I20260929 02:23:20.095914 27768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:20.108652 27768 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:20.109989 27768 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:20.119829 27768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: Generated new TSK 0
I20260929 02:23:20.120646 27768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:20.125427 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.1:0
--local_ip_for_outbound_sockets=127.7.170.1
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--builtin_ntp_servers=127.7.170.20:41377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260929 02:23:20.447248 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:20.447539 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:20.447660 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:20.455008 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:20.455104 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:20.455246 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.1
I20260929 02:23:20.464217 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.170.1
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.1
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:20.465483 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:20.467244 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:20.479868 27787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:20.480170 27786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:20.479905 27789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:20.480574 27781 server_base.cc:1034] running on GCE node
I20260929 02:23:20.481150 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:20.482067 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:20.483280 27781 hybrid_clock.cc:648] HybridClock initialized: now 1790648600483229 us; error 36 us; skew 500 ppm
I20260929 02:23:20.483592 27781 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:20.485807 27781 webserver.cc:460] Webserver started at http://127.7.170.1:42721/ using document root <none> and password file <none>
I20260929 02:23:20.486488 27781 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:20.486577 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:20.486922 27781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:20.489804 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data/instance:
uuid: "1d44f650825748359f11477706f254cc"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "53f8f3c556bc9ddc4453ab3f5db844a1"
server_key_iv: "282888ece3868911a725c0f511c24a08"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.490551 27781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance:
uuid: "1d44f650825748359f11477706f254cc"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "53f8f3c556bc9ddc4453ab3f5db844a1"
server_key_iv: "282888ece3868911a725c0f511c24a08"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.495429 27781 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:20.498735 27795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.500021 27781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:23:20.500159 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal
uuid: "1d44f650825748359f11477706f254cc"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "53f8f3c556bc9ddc4453ab3f5db844a1"
server_key_iv: "282888ece3868911a725c0f511c24a08"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.500351 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:20.513916 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:20.514634 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:20.514871 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:20.515763 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:20.517205 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:20.517309 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.517400 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:20.517455 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.546042 27781 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.1:36265
I20260929 02:23:20.546120 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.1:36265 every 8 connection(s)
I20260929 02:23:20.547261 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/data/info.pb
I20260929 02:23:20.549981 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27781
I20260929 02:23:20.550134 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal/instance
I20260929 02:23:20.550526 7848 external_mini_cluster.cc:1428] Setting key 79d2d9ef7c96b7f66e79811577926e8b
I20260929 02:23:20.553061 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.2:0
--local_ip_for_outbound_sockets=127.7.170.2
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--builtin_ntp_servers=127.7.170.20:41377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:20.559420 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:20.549591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.1:34839 (local address 127.7.170.62:34841)
0929 02:23:20.549792 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:20.549798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:20.551273 (+ 1475us) server_negotiation.cc:407] Connection header received
0929 02:23:20.553096 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:20.553102 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:20.553187 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:20.553294 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:20.554727 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.555416 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.556952 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.557363 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.557472 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:20.558683 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:20.558689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:20.558693 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:20.558709 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:20.558738 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:20.558749 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:20.558827 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:20.559023 (+ 196us) server_negotiation.cc:299] Negotiation successful
0929 02:23:20.559117 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:20.560395 27910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:20.550092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34841 (local address 127.7.170.1:34839)
0929 02:23:20.551119 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:20.551168 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:20.552808 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:20.553673 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:20.553685 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:20.553795 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:20.554549 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:20.554575 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.555565 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.555569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:20.556786 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:20.556798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.557571 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.557577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:20.557785 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:20.558401 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:20.558404 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:20.558439 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:20.558508 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:20.558839 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:20.558851 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:20.558858 (+ 7us) client_negotiation.cc:769] Sending connection context
0929 02:23:20.559000 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 02:23:20.559359 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":327,"threads_started":1}
I20260929 02:23:20.561897 27908 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34841
I20260929 02:23:20.562179 27908 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:20.563195 27908 heartbeater.cc:511] Master 127.7.170.62:34841 requested a full tablet report, sending...
I20260929 02:23:20.565220 27722 ts_manager.cc:194] Registered new tserver with Master: 1d44f650825748359f11477706f254cc (127.7.170.1:36265)
I20260929 02:23:20.566646 27722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.1:34839
W20260929 02:23:20.871170 27911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:20.871425 27911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:20.871548 27911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:20.878706 27911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:20.878795 27911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:20.878934 27911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.2
I20260929 02:23:20.887925 27911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.170.2
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.2
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:20.889176 27911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:20.890851 27911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:20.902909 27917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:20.902935 27919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:20.902966 27916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:20.903688 27911 server_base.cc:1034] running on GCE node
I20260929 02:23:20.904260 27911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:20.905238 27911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:20.906424 27911 hybrid_clock.cc:648] HybridClock initialized: now 1790648600906376 us; error 47 us; skew 500 ppm
I20260929 02:23:20.906704 27911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:20.908869 27911 webserver.cc:460] Webserver started at http://127.7.170.2:34435/ using document root <none> and password file <none>
I20260929 02:23:20.909552 27911 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:20.909639 27911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:20.909932 27911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:20.912664 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data/instance:
uuid: "8f1ca60ff4b74f098b38f6fe90a16b55"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "ba6a66ca6d6afda9b7b0ccf2e7f3fcba"
server_key_iv: "1d1676356788c1453f2c0c08ccac67cd"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.913435 27911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance:
uuid: "8f1ca60ff4b74f098b38f6fe90a16b55"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "ba6a66ca6d6afda9b7b0ccf2e7f3fcba"
server_key_iv: "1d1676356788c1453f2c0c08ccac67cd"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.918239 27911 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260929 02:23:20.921620 27925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.922857 27911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:20.923005 27911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal
uuid: "8f1ca60ff4b74f098b38f6fe90a16b55"
format_stamp: "Formatted at 2026-09-29 02:23:20 on dist-test-slave-8cs2"
server_key: "ba6a66ca6d6afda9b7b0ccf2e7f3fcba"
server_key_iv: "1d1676356788c1453f2c0c08ccac67cd"
server_key_version: "encryptionkey@0"
I20260929 02:23:20.923194 27911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:20.940423 27911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:20.941376 27911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:20.941681 27911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:20.942904 27911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:20.944355 27911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:20.944451 27911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.944554 27911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:20.944623 27911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:20.973455 27911 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.2:36279
I20260929 02:23:20.973517 28037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.2:36279 every 8 connection(s)
I20260929 02:23:20.974735 27911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/data/info.pb
I20260929 02:23:20.979749 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 27911
I20260929 02:23:20.979910 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal/instance
I20260929 02:23:20.980418 7848 external_mini_cluster.cc:1428] Setting key 90404ce04740d7839d9ae6d8cdd9d690
I20260929 02:23:20.983173 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.170.3:0
--local_ip_for_outbound_sockets=127.7.170.3
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--builtin_ntp_servers=127.7.170.20:41377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260929 02:23:20.988027 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:20.977156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:32851 (local address 127.7.170.62:34841)
0929 02:23:20.977356 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:20.977362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:20.978966 (+ 1604us) server_negotiation.cc:407] Connection header received
0929 02:23:20.981234 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:20.981239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:20.981321 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:20.981405 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:20.982567 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.983327 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.985461 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.985916 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.986022 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:20.987383 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:20.987389 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:20.987392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:20.987410 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:20.987440 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:20.987452 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:20.987517 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:20.987718 (+ 201us) server_negotiation.cc:299] Negotiation successful
0929 02:23:20.987793 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:23:20.989351 28040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:20.977745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34841 (local address 127.7.170.2:32851)
0929 02:23:20.978792 (+ 1047us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:20.978848 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:20.980907 (+ 2059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:20.981647 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:20.981656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:20.981749 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:20.982384 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:20.982408 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.983478 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.983483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:20.985325 (+ 1842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:20.985339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:20.986122 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:20.986128 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:20.986385 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:20.987069 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:20.987072 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:20.987121 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:20.987209 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:20.987533 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:20.987542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:20.987550 (+ 8us) client_negotiation.cc:769] Sending connection context
0929 02:23:20.987717 (+ 167us) client_negotiation.cc:240] Negotiation successful
0929 02:23:20.988099 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":354,"threads_started":1}
I20260929 02:23:20.990708 28038 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34841
I20260929 02:23:20.991004 28038 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:20.991748 28038 heartbeater.cc:511] Master 127.7.170.62:34841 requested a full tablet report, sending...
I20260929 02:23:20.993204 27722 ts_manager.cc:194] Registered new tserver with Master: 8f1ca60ff4b74f098b38f6fe90a16b55 (127.7.170.2:36279)
I20260929 02:23:20.994287 27722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.2:32851
W20260929 02:23:21.305778 28041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:21.306207 28041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:21.306358 28041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:21.314332 28041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:21.314424 28041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:21.314531 28041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.170.3
I20260929 02:23:21.323522 28041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:41377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.170.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.170.3
--webserver_port=0
--tserver_master_addrs=127.7.170.62:34841
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.170.3
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:21.324698 28041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:21.326514 28041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:21.338800 28046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:21.338899 28047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:21.338876 28049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:21.339782 28041 server_base.cc:1034] running on GCE node
I20260929 02:23:21.340312 28041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:21.341208 28041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:21.342397 28041 hybrid_clock.cc:648] HybridClock initialized: now 1790648601342357 us; error 37 us; skew 500 ppm
I20260929 02:23:21.342671 28041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260929 02:23:21.344718 28041 webserver.cc:460] Webserver started at http://127.7.170.3:38937/ using document root <none> and password file <none>
I20260929 02:23:21.345371 28041 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:21.345458 28041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:21.345748 28041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:21.348402 28041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data/instance:
uuid: "5a2d25bd470048cb9bdd58daa417bb8f"
format_stamp: "Formatted at 2026-09-29 02:23:21 on dist-test-slave-8cs2"
server_key: "3d1455f7def6fb697cdfbd94f53af133"
server_key_iv: "95fe31dfe7b5bb03f9d35ed9ff00bb8b"
server_key_version: "encryptionkey@0"
I20260929 02:23:21.349218 28041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance:
uuid: "5a2d25bd470048cb9bdd58daa417bb8f"
format_stamp: "Formatted at 2026-09-29 02:23:21 on dist-test-slave-8cs2"
server_key: "3d1455f7def6fb697cdfbd94f53af133"
server_key_iv: "95fe31dfe7b5bb03f9d35ed9ff00bb8b"
server_key_version: "encryptionkey@0"
I20260929 02:23:21.354028 28041 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:23:21.357219 28055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:21.358400 28041 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260929 02:23:21.358537 28041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal
uuid: "5a2d25bd470048cb9bdd58daa417bb8f"
format_stamp: "Formatted at 2026-09-29 02:23:21 on dist-test-slave-8cs2"
server_key: "3d1455f7def6fb697cdfbd94f53af133"
server_key_iv: "95fe31dfe7b5bb03f9d35ed9ff00bb8b"
server_key_version: "encryptionkey@0"
I20260929 02:23:21.358716 28041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:21.378178 28041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:21.378937 28041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:21.379196 28041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:21.380167 28041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260929 02:23:21.381587 28041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260929 02:23:21.381677 28041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:21.381783 28041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260929 02:23:21.381848 28041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:21.411753 28041 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.3:36279
I20260929 02:23:21.411845 28167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.3:36279 every 8 connection(s)
I20260929 02:23:21.413067 28041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/data/info.pb
I20260929 02:23:21.421244 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28041
I20260929 02:23:21.421411 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal/instance
I20260929 02:23:21.421867 7848 external_mini_cluster.cc:1428] Setting key 173e7fddf4dcd14356f597bedf10db19
I20260929 02:23:21.425323 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.415443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.3:42055 (local address 127.7.170.62:34841)
0929 02:23:21.415595 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.415599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.416895 (+ 1296us) server_negotiation.cc:407] Connection header received
0929 02:23:21.418654 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.418658 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.418730 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.418818 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:21.419995 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.420724 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.422598 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.423141 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.423263 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.424595 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:21.424601 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:21.424605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:21.424626 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:21.424666 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:21.424681 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:21.424762 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.425021 (+ 259us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.425098 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260929 02:23:21.426445 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.415913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:34841 (local address 127.7.170.3:42055)
0929 02:23:21.416757 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.416800 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.418400 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.419076 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.419085 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.419182 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:21.419784 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.419809 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.420963 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.420968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.422456 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.422472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.423355 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.423361 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.423634 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.424293 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0929 02:23:21.424295 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:21.424336 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0929 02:23:21.424428 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:21.424797 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:21.424814 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:21.424820 (+ 6us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.425012 (+ 192us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.425307 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":258,"threads_started":1}
I20260929 02:23:21.427716 28168 heartbeater.cc:347] Connected to a master server at 127.7.170.62:34841
I20260929 02:23:21.428014 28168 heartbeater.cc:464] Registering TS with master...
I20260929 02:23:21.428721 28168 heartbeater.cc:511] Master 127.7.170.62:34841 requested a full tablet report, sending...
I20260929 02:23:21.430011 27722 ts_manager.cc:194] Registered new tserver with Master: 5a2d25bd470048cb9bdd58daa417bb8f (127.7.170.3:36279)
I20260929 02:23:21.430785 27722 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.170.3:42055
I20260929 02:23:21.436851 7848 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260929 02:23:21.445986 27760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.440698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.7.170.62:34841)
0929 02:23:21.440852 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.440856 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.441198 (+ 342us) server_negotiation.cc:407] Connection header received
0929 02:23:21.441311 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.441315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.441390 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.441474 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:21.441986 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.442691 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.444019 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.444415 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.444501 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.445420 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:21.445425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:21.445428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:21.445444 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:21.445473 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:21.445485 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:21.445550 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.445648 (+ 98us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.445716 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260929 02:23:21.449395 27722 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260929 02:23:21.456041 27722 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260929 02:23:21.478868 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.467501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:36265 (local address 127.0.0.1:47548)
0929 02:23:21.468333 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.468376 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.468541 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.470357 (+ 1816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.470363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.470389 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:21.470724 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.470733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.472644 (+ 1911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.472650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.475167 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.475183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.477884 (+ 2701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.477890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.478125 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.478180 (+ 55us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.478309 (+ 129us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.478516 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":1186,"threads_started":1}
I20260929 02:23:21.479794 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.468372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.7.170.1:36265)
0929 02:23:21.469720 (+ 1348us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.469727 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.469759 (+ 32us) server_negotiation.cc:407] Connection header received
0929 02:23:21.469879 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.469887 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.470139 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.470343 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:21.470960 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.472471 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.475393 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.477359 (+ 1966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.477497 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.477755 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.479277 (+ 1522us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.479464 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1203,"thread_start_us":1067,"threads_started":1}
I20260929 02:23:21.480470 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.468036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.7.170.2:36279)
0929 02:23:21.469326 (+ 1290us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.469336 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.471207 (+ 1871us) server_negotiation.cc:407] Connection header received
0929 02:23:21.471317 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.471325 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.471568 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.471742 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:21.472665 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.474301 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.476992 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.478469 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.478594 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.478883 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.479927 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.480105 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108,"thread_start_us":1189,"threads_started":1}
I20260929 02:23:21.479777 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.467584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.2:36279 (local address 127.0.0.1:40896)
0929 02:23:21.470967 (+ 3383us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.470983 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.471087 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.472025 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.472031 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.472055 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:21.472489 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.472500 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.474473 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.474479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.476751 (+ 2272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.476766 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.479009 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.479016 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.479203 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.479237 (+ 34us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.479317 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.479457 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3269,"thread_start_us":3094,"threads_started":1}
I20260929 02:23:21.484916 27843 tablet_service.cc:1467] Processing CreateTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (DEFAULT_TABLE table=test-table [id=618624f508574495bc5afcffb98f32f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:21.486509 27843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:21.489161 27973 tablet_service.cc:1467] Processing CreateTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (DEFAULT_TABLE table=test-table [id=618624f508574495bc5afcffb98f32f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:21.490653 27973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:21.491053 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.471039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36279 (local address 127.0.0.1:44360)
0929 02:23:21.481077 (+ 10038us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.481097 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.481197 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.485748 (+ 4551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.485753 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.485780 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:21.485963 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.485973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.487474 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.487479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.488604 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.488616 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.490304 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.490310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.490519 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.490552 (+ 33us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.490632 (+ 80us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.490771 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9913,"thread_start_us":14091,"threads_started":1}
I20260929 02:23:21.491647 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.471372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.7.170.3:36279)
0929 02:23:21.485144 (+ 13772us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.485153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.485190 (+ 37us) server_negotiation.cc:407] Connection header received
0929 02:23:21.485289 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.485297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.485541 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.485732 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:21.486094 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.487360 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.488749 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.489677 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.489793 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.490057 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.491260 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.491397 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13381,"thread_start_us":13571,"threads_started":1}
I20260929 02:23:21.495800 28103 tablet_service.cc:1467] Processing CreateTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (DEFAULT_TABLE table=test-table [id=618624f508574495bc5afcffb98f32f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260929 02:23:21.497140 28103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:21.501500 28186 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Bootstrap starting.
I20260929 02:23:21.504736 28187 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Bootstrap starting.
I20260929 02:23:21.505415 28186 tablet_bootstrap.cc:654] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:21.506773 28186 log.cc:824] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:21.507997 28187 tablet_bootstrap.cc:654] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:21.508944 28187 log.cc:824] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:21.511281 28190 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Bootstrap starting.
I20260929 02:23:21.511960 28186 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: No bootstrap required, opened a new log
I20260929 02:23:21.512203 28186 ts_tablet_manager.cc:1397] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20260929 02:23:21.515045 28190 tablet_bootstrap.cc:654] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:21.516076 28190 log.cc:824] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:21.517915 28187 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: No bootstrap required, opened a new log
I20260929 02:23:21.518187 28187 ts_tablet_manager.cc:1397] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.007s
I20260929 02:23:21.519619 28190 tablet_bootstrap.cc:492] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: No bootstrap required, opened a new log
I20260929 02:23:21.519871 28190 ts_tablet_manager.cc:1397] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260929 02:23:21.519901 28186 raft_consensus.cc:348] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.520268 28186 raft_consensus.cc:374] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:21.520372 28186 raft_consensus.cc:729] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d44f650825748359f11477706f254cc, State: Initialized, Role: FOLLOWER
I20260929 02:23:21.520910 28186 consensus_queue.cc:260] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.521943 27908 heartbeater.cc:503] Master 127.7.170.62:34841 was elected leader, sending a full tablet report...
I20260929 02:23:21.522552 28186 ts_tablet_manager.cc:1428] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Time spent starting tablet: real 0.010s user 0.006s sys 0.003s
I20260929 02:23:21.528751 28187 raft_consensus.cc:348] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.529253 28187 raft_consensus.cc:374] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:21.529362 28187 raft_consensus.cc:729] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f1ca60ff4b74f098b38f6fe90a16b55, State: Initialized, Role: FOLLOWER
I20260929 02:23:21.529987 28187 consensus_queue.cc:260] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.529973 28190 raft_consensus.cc:348] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.530400 28190 raft_consensus.cc:374] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:21.530555 28190 raft_consensus.cc:729] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a2d25bd470048cb9bdd58daa417bb8f, State: Initialized, Role: FOLLOWER
I20260929 02:23:21.531421 28038 heartbeater.cc:503] Master 127.7.170.62:34841 was elected leader, sending a full tablet report...
I20260929 02:23:21.531767 28187 ts_tablet_manager.cc:1428] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260929 02:23:21.531162 28190 consensus_queue.cc:260] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.532883 28168 heartbeater.cc:503] Master 127.7.170.62:34841 was elected leader, sending a full tablet report...
I20260929 02:23:21.533535 28190 ts_tablet_manager.cc:1428] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
W20260929 02:23:21.549258 27909 tablet.cc:2369] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:21.561170 28193 raft_consensus.cc:482] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260929 02:23:21.561429 28193 raft_consensus.cc:504] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.562950 28193 leader_election.cc:288] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1d44f650825748359f11477706f254cc (127.7.170.1:36265), 5a2d25bd470048cb9bdd58daa417bb8f (127.7.170.3:36279)
I20260929 02:23:21.571171 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.563488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:50841 (local address 127.7.170.3:36279)
0929 02:23:21.563685 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.563692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.564418 (+ 726us) server_negotiation.cc:407] Connection header received
0929 02:23:21.564555 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.564559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.564622 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.564704 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:21.565332 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.566180 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.569010 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.570182 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.570292 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.570381 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.570817 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.570941 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:21.571171 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.563343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.170.2:56117 (local address 127.7.170.1:36265)
0929 02:23:21.563546 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.563552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.564274 (+ 722us) server_negotiation.cc:407] Connection header received
0929 02:23:21.564446 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.564451 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.564533 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.564606 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0929 02:23:21.565203 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.566094 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.568994 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.570160 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.570287 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.570381 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.570817 (+ 436us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.570941 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:23:21.571334 28195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.563450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.1:36265 (local address 127.7.170.2:56117)
0929 02:23:21.564163 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.564184 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.564320 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.564633 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.564637 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.564660 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:21.565064 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.565075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.566212 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.566217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.568775 (+ 2558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.568784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.570388 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.570395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.570659 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.570706 (+ 47us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.570772 (+ 66us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.570965 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":240,"thread_start_us":355,"threads_started":1}
I20260929 02:23:21.571362 28196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.563670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.3:36279 (local address 127.7.170.2:50841)
0929 02:23:21.564323 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:21.564336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:21.564452 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:21.564735 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:21.564738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:21.564768 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0929 02:23:21.565191 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.565201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.566305 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.566317 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.568765 (+ 2448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:21.568779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.570399 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.570406 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:21.570639 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.570685 (+ 46us) client_negotiation.cc:769] Sending connection context
0929 02:23:21.570772 (+ 87us) client_negotiation.cc:240] Negotiation successful
0929 02:23:21.570965 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":199,"thread_start_us":374,"threads_started":1}
I20260929 02:23:21.572333 27863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e9250a826349aeb6ceb4ea9bc7b4f1" candidate_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d44f650825748359f11477706f254cc" is_pre_election: true
I20260929 02:23:21.572512 28123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e9250a826349aeb6ceb4ea9bc7b4f1" candidate_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" is_pre_election: true
I20260929 02:23:21.572865 27863 raft_consensus.cc:2454] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1ca60ff4b74f098b38f6fe90a16b55 in term 0.
I20260929 02:23:21.573028 28123 raft_consensus.cc:2454] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f1ca60ff4b74f098b38f6fe90a16b55 in term 0.
I20260929 02:23:21.573540 27926 leader_election.cc:302] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d44f650825748359f11477706f254cc, 8f1ca60ff4b74f098b38f6fe90a16b55; no voters:
I20260929 02:23:21.573939 28193 raft_consensus.cc:2781] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260929 02:23:21.574065 28193 raft_consensus.cc:482] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260929 02:23:21.574165 28193 raft_consensus.cc:3037] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:21.576274 28193 raft_consensus.cc:504] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.576825 28193 leader_election.cc:288] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [CANDIDATE]: Term 1 election: Requested vote from peers 1d44f650825748359f11477706f254cc (127.7.170.1:36265), 5a2d25bd470048cb9bdd58daa417bb8f (127.7.170.3:36279)
I20260929 02:23:21.577239 27863 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e9250a826349aeb6ceb4ea9bc7b4f1" candidate_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1d44f650825748359f11477706f254cc"
I20260929 02:23:21.577286 28123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25e9250a826349aeb6ceb4ea9bc7b4f1" candidate_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a2d25bd470048cb9bdd58daa417bb8f"
I20260929 02:23:21.577453 27863 raft_consensus.cc:3037] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:21.577491 28123 raft_consensus.cc:3037] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:21.580703 28123 raft_consensus.cc:2454] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1ca60ff4b74f098b38f6fe90a16b55 in term 1.
I20260929 02:23:21.580703 27863 raft_consensus.cc:2454] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f1ca60ff4b74f098b38f6fe90a16b55 in term 1.
I20260929 02:23:21.581138 27926 leader_election.cc:302] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1d44f650825748359f11477706f254cc, 5a2d25bd470048cb9bdd58daa417bb8f, 8f1ca60ff4b74f098b38f6fe90a16b55; no voters:
I20260929 02:23:21.581418 28193 raft_consensus.cc:2781] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:21.581851 28193 raft_consensus.cc:686] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 1 LEADER]: Becoming Leader. State: Replica: 8f1ca60ff4b74f098b38f6fe90a16b55, State: Running, Role: LEADER
I20260929 02:23:21.582296 28193 consensus_queue.cc:237] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } }
I20260929 02:23:21.586877 27722 catalog_manager.cc:5495] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f1ca60ff4b74f098b38f6fe90a16b55 (127.7.170.2). New cstate: current_term: 1 leader_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f1ca60ff4b74f098b38f6fe90a16b55" member_type: VOTER last_known_addr { host: "127.7.170.2" port: 36279 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 } health_report { overall_health: UNKNOWN } } }
I20260929 02:23:21.601576 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.596435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40910 (local address 127.7.170.2:36279)
0929 02:23:21.596621 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.596627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.596685 (+ 58us) server_negotiation.cc:407] Connection header received
0929 02:23:21.596870 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.596874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.596981 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.597102 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:21.597417 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.598138 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.599501 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.599967 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.600091 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.600510 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:21.601065 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:21.601196 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.601270 (+ 74us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.601336 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:23:21.609805 28123 raft_consensus.cc:1261] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 1 FOLLOWER]: Refusing update from remote peer 8f1ca60ff4b74f098b38f6fe90a16b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:21.609805 27863 raft_consensus.cc:1261] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 1 FOLLOWER]: Refusing update from remote peer 8f1ca60ff4b74f098b38f6fe90a16b55: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:21.610718 28197 consensus_queue.cc:1035] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a2d25bd470048cb9bdd58daa417bb8f" member_type: VOTER last_known_addr { host: "127.7.170.3" port: 36279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:21.611065 28193 consensus_queue.cc:1035] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d44f650825748359f11477706f254cc" member_type: VOTER last_known_addr { host: "127.7.170.1" port: 36265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:21.635070 27722 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:21.635304 27722 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260929 02:23:21.638545 27722 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9729b135dd741a0881f06b7414a0cc8: Sending DeleteTablet for 3 replicas of tablet 25e9250a826349aeb6ceb4ea9bc7b4f1
I20260929 02:23:21.639739 27843 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:21 UTC) from {username='slave'} at 127.0.0.1:47548
I20260929 02:23:21.639928 27973 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:21 UTC) from {username='slave'} at 127.0.0.1:40896
I20260929 02:23:21.640586 28103 tablet_service.cc:1514] Processing DeleteTablet for tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-29 02:23:21 UTC) from {username='slave'} at 127.0.0.1:44360
I20260929 02:23:21.640697 28211 tablet_replica.cc:330] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: stopping tablet replica
I20260929 02:23:21.640916 28212 tablet_replica.cc:330] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: stopping tablet replica
I20260929 02:23:21.641139 28211 raft_consensus.cc:2229] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:21.641286 28212 raft_consensus.cc:2229] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 1 LEADER]: Raft consensus shutting down.
I20260929 02:23:21.641399 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27781
I20260929 02:23:21.641618 28211 raft_consensus.cc:2258] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:21.641891 28212 raft_consensus.cc:2258] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:21.643574 28211 ts_tablet_manager.cc:1905] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:21.643796 28212 ts_tablet_manager.cc:1905] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:21.650432 28211 ts_tablet_manager.cc:1918] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:21.650586 28211 log.cc:1196] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-0/wal/wals/25e9250a826349aeb6ceb4ea9bc7b4f1
I20260929 02:23:21.650775 28212 ts_tablet_manager.cc:1918] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:21.650925 28212 log.cc:1196] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-1/wal/wals/25e9250a826349aeb6ceb4ea9bc7b4f1
I20260929 02:23:21.651105 28211 ts_tablet_manager.cc:1939] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 1d44f650825748359f11477706f254cc: Deleting consensus metadata
I20260929 02:23:21.651470 28212 ts_tablet_manager.cc:1939] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 8f1ca60ff4b74f098b38f6fe90a16b55: Deleting consensus metadata
I20260929 02:23:21.652872 27706 catalog_manager.cc:4841] TS 1d44f650825748359f11477706f254cc (127.7.170.1:36265): tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (table test-table [id=618624f508574495bc5afcffb98f32f1]) successfully deleted
I20260929 02:23:21.653025 27709 catalog_manager.cc:4841] TS 8f1ca60ff4b74f098b38f6fe90a16b55 (127.7.170.2:36279): tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (table test-table [id=618624f508574495bc5afcffb98f32f1]) successfully deleted
I20260929 02:23:21.657455 28213 tablet_replica.cc:330] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: stopping tablet replica
I20260929 02:23:21.657881 28213 raft_consensus.cc:2229] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260929 02:23:21.658389 28213 raft_consensus.cc:2258] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260929 02:23:21.660305 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.643917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47558 (local address 127.7.170.1:36265)
0929 02:23:21.644082 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.644088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.653593 (+ 9505us) server_negotiation.cc:407] Connection header received
0929 02:23:21.653760 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.653765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.653866 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.653956 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:21.654421 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.655511 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.657391 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.657824 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.657938 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.659060 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:21.659068 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:21.659080 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:21.659585 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:21.659632 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:21.659650 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:21.659745 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.659872 (+ 127us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.659981 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:23:21.660454 28213 ts_tablet_manager.cc:1905] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260929 02:23:21.660897 27903 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:21.668277 28213 ts_tablet_manager.cc:1918] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260929 02:23:21.668444 28213 log.cc:1196] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Deleting WAL directory at /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790648296482495-7848-0/minicluster-data/ts-2/wal/wals/25e9250a826349aeb6ceb4ea9bc7b4f1
I20260929 02:23:21.668949 28213 ts_tablet_manager.cc:1939] T 25e9250a826349aeb6ceb4ea9bc7b4f1 P 5a2d25bd470048cb9bdd58daa417bb8f: Deleting consensus metadata
I20260929 02:23:21.670748 27709 catalog_manager.cc:4841] TS 5a2d25bd470048cb9bdd58daa417bb8f (127.7.170.3:36279): tablet 25e9250a826349aeb6ceb4ea9bc7b4f1 (table test-table [id=618624f508574495bc5afcffb98f32f1]) successfully deleted
I20260929 02:23:21.747958 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27781
I20260929 02:23:21.768862 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27911
I20260929 02:23:21.775628 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.769590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.7.170.2:36279)
0929 02:23:21.769744 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.769749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.769883 (+ 134us) server_negotiation.cc:407] Connection header received
0929 02:23:21.770055 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.770061 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.770143 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.770230 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:21.770604 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.771361 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.772569 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.773129 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.773233 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.774275 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:21.774284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:21.774298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:21.774886 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:21.774947 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:21.774968 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:21.775056 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.775205 (+ 149us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.775311 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260929 02:23:21.776259 28033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:21.884532 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27911
I20260929 02:23:21.899782 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28041
I20260929 02:23:21.906354 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:21.900562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52262 (local address 127.7.170.3:36279)
0929 02:23:21.900740 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:21.900746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:21.900766 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:23:21.900862 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:21.900867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:21.901000 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:21.901088 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:21.901489 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.902255 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.903511 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:21.903915 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:21.904023 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:21.905042 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:21.905050 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:21.905065 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:21.905638 (+ 573us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:21.905686 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:21.905706 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:21.905813 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:21.905943 (+ 130us) server_negotiation.cc:299] Negotiation successful
0929 02:23:21.906052 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:23:21.906989 28163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:21.993696 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28041
I20260929 02:23:22.007684 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 27691
I20260929 02:23:22.008773 27752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:22.115219 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 27691
2026-09-29T02:23:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2550 ms)
[----------] 6 tests from EncryptionPolicyTest (20967 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:22 dist-test-slave-8cs2 krb5kdc[28221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:22 dist-test-slave-8cs2 krb5kdc[28221](info): set up 2 sockets
Sep 29 02:23:22 dist-test-slave-8cs2 krb5kdc[28221](info): commencing operation
krb5kdc: starting...
W20260929 02:23:23.145488 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:23 dist-test-slave-8cs2 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:23.402580 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33275
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:23.721396 28234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:23.721691 28234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:23.721836 28234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:23.728915 28234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:23.729053 28234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:23.729125 28234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:23.729200 28234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:23.729255 28234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:23.738849 28234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33275
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:23.740067 28234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:23.741328 28234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:23.752139 28239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:23.752142 28242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:23.752168 28240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:23.753185 28234 server_base.cc:1034] running on GCE node
I20260929 02:23:23.753893 28234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:23.755160 28234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:23.756369 28234 hybrid_clock.cc:648] HybridClock initialized: now 1790648603756324 us; error 36 us; skew 500 ppm
I20260929 02:23:23.756676 28234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:23 dist-test-slave-8cs2 krb5kdc[28221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648603, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:23.759646 28234 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:23.761010 28234 webserver.cc:460] Webserver started at http://127.0.0.1:41657/ using document root <none> and password file <none>
I20260929 02:23:23.761664 28234 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:23.761773 28234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:23.762042 28234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:23.764842 28234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "77b6297096ad49ccbf7d463b86d98837"
format_stamp: "Formatted at 2026-09-29 02:23:23 on dist-test-slave-8cs2"
server_key: "094d10d41c0a3e4fa08491279ccc4943"
server_key_iv: "c70377a38e87317076a18dc89442abd5"
server_key_version: "encryptionkey@0"
I20260929 02:23:23.765591 28234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "77b6297096ad49ccbf7d463b86d98837"
format_stamp: "Formatted at 2026-09-29 02:23:23 on dist-test-slave-8cs2"
server_key: "094d10d41c0a3e4fa08491279ccc4943"
server_key_iv: "c70377a38e87317076a18dc89442abd5"
server_key_version: "encryptionkey@0"
I20260929 02:23:23.770390 28234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:23:23.773604 28249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:23.774868 28234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:23.775033 28234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "77b6297096ad49ccbf7d463b86d98837"
format_stamp: "Formatted at 2026-09-29 02:23:23 on dist-test-slave-8cs2"
server_key: "094d10d41c0a3e4fa08491279ccc4943"
server_key_iv: "c70377a38e87317076a18dc89442abd5"
server_key_version: "encryptionkey@0"
I20260929 02:23:23.775204 28234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:23.832597 28234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:23.836894 28234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:23.837176 28234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:23.853730 28234 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33275
I20260929 02:23:23.853760 28300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33275 every 8 connection(s)
I20260929 02:23:23.855013 28234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:23.857592 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28234
I20260929 02:23:23.857802 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:23.858589 7848 external_mini_cluster.cc:1428] Setting key 23673afe362014658aaebb0db6e66369
I20260929 02:23:23.859143 28301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:23.872650 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837: Bootstrap starting.
Sep 29 02:23:23 dist-test-slave-8cs2 krb5kdc[28221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648603, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:23:23.875650 28301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:23.876672 28301 log.cc:824] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:23.879627 28301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837: No bootstrap required, opened a new log
I20260929 02:23:23.881096 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:23.860391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48087 (local address 127.0.0.1:33275)
0929 02:23:23.861138 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:23.861149 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:23.861188 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:23:23.863471 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:23.863491 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:23.863854 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:23.864180 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:23.865913 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:23.867187 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:23.868914 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:23.869628 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:23.869752 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:23.874161 (+ 4409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:23.874169 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:23.874186 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:23.874219 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:23.877067 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:23.877820 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:23.877826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:23.877832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:23.878007 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:23.878455 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:23.878459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:23.878461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:23.878950 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:23.879236 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:23.879713 (+ 477us) server_negotiation.cc:299] Negotiation successful
0929 02:23:23.879998 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":323,"threads_started":1}
I20260929 02:23:23.884891 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:23.887744 28301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } }
I20260929 02:23:23.888104 28301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:23.888206 28301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77b6297096ad49ccbf7d463b86d98837, State: Initialized, Role: FOLLOWER
I20260929 02:23:23.888738 28301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } }
I20260929 02:23:23.888993 28301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:23.889128 28301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:23.889298 28301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:23.891836 28301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } }
I20260929 02:23:23.892414 28301 leader_election.cc:302] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77b6297096ad49ccbf7d463b86d98837; no voters:
I20260929 02:23:23.893033 28301 leader_election.cc:288] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:23.893167 28311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:23.893740 28311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [term 1 LEADER]: Becoming Leader. State: Replica: 77b6297096ad49ccbf7d463b86d98837, State: Running, Role: LEADER
I20260929 02:23:23.894354 28311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } }
I20260929 02:23:23.896276 28301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:23.897890 28312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77b6297096ad49ccbf7d463b86d98837. Latest consensus state: current_term: 1 leader_uuid: "77b6297096ad49ccbf7d463b86d98837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } } }
I20260929 02:23:23.898341 28312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:23.905581 28317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:23.906019 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:23.889945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58429 (local address 127.0.0.1:33275)
0929 02:23:23.890124 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:23.890130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:23.890153 (+ 23us) server_negotiation.cc:407] Connection header received
0929 02:23:23.890252 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:23.890260 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:23.890340 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:23.890453 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:23.891744 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:23.892834 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:23.894515 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:23.895195 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:23.895306 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:23.898176 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:23.898185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:23.898191 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:23.898229 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:23.901935 (+ 3706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:23.902949 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:23.902954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:23.902958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:23.903111 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:23.903657 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:23.903662 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:23.903666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:23.904090 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:23.904381 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:23.905412 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0929 02:23:23.905543 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260929 02:23:23.905846 28313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77b6297096ad49ccbf7d463b86d98837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77b6297096ad49ccbf7d463b86d98837" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33275 } } }
I20260929 02:23:23.906762 28313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:23.911283 28317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:23.920661 28317 catalog_manager.cc:1347] Generated new cluster ID: c8afffd07e004d7fb5b4e0eae3ce7327
I20260929 02:23:23.920758 28317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:23.937072 28317 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:23.938301 28317 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:23.949034 28317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77b6297096ad49ccbf7d463b86d98837: Generated new TSK 0
I20260929 02:23:23.949888 28317 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:23.964823 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28234
I20260929 02:23:23.966452 28296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:24.057962 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28234
W20260929 02:23:24.072678 28219 connection.cc:535] client connection to 127.0.0.1:33275 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33275): Connection reset by peer (error 104)
2026-09-29T02:23:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1939 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:24 dist-test-slave-8cs2 krb5kdc[28333](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:24 dist-test-slave-8cs2 krb5kdc[28333](info): set up 2 sockets
Sep 29 02:23:24 dist-test-slave-8cs2 krb5kdc[28333](info): commencing operation
krb5kdc: starting...
W20260929 02:23:25.081290 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.990s user 0.001s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:25 dist-test-slave-8cs2 krb5kdc[28333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648605, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:25.341584 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:25.660307 28346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:25.660593 28346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:25.660720 28346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:25.667980 28346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:25.668092 28346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:25.668164 28346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:25.668246 28346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:25.668303 28346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:25.678146 28346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44131
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:25.679419 28346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:25.680694 28346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:25.692253 28352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:25.692229 28351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:25.692466 28354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:25.693115 28346 server_base.cc:1034] running on GCE node
I20260929 02:23:25.693830 28346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:25.695237 28346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:25.696442 28346 hybrid_clock.cc:648] HybridClock initialized: now 1790648605696385 us; error 47 us; skew 500 ppm
I20260929 02:23:25.696821 28346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:25 dist-test-slave-8cs2 krb5kdc[28333](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:25.700052 28346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:25.701522 28346 webserver.cc:460] Webserver started at http://127.0.0.1:34351/ using document root <none> and password file <none>
I20260929 02:23:25.702183 28346 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:25.702281 28346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:25.702529 28346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:25.705528 28346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "1a746009f4fa484bb7395b990553d9b5"
format_stamp: "Formatted at 2026-09-29 02:23:25 on dist-test-slave-8cs2"
server_key: "e3868b975c0b8709f5bcbeb796864f9d"
server_key_iv: "b7e46eb5bba9d629f55951cfcbd9a3a5"
server_key_version: "encryptionkey@0"
I20260929 02:23:25.706275 28346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "1a746009f4fa484bb7395b990553d9b5"
format_stamp: "Formatted at 2026-09-29 02:23:25 on dist-test-slave-8cs2"
server_key: "e3868b975c0b8709f5bcbeb796864f9d"
server_key_iv: "b7e46eb5bba9d629f55951cfcbd9a3a5"
server_key_version: "encryptionkey@0"
I20260929 02:23:25.711377 28346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:25.714839 28361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:25.716183 28346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:25.716351 28346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "1a746009f4fa484bb7395b990553d9b5"
format_stamp: "Formatted at 2026-09-29 02:23:25 on dist-test-slave-8cs2"
server_key: "e3868b975c0b8709f5bcbeb796864f9d"
server_key_iv: "b7e46eb5bba9d629f55951cfcbd9a3a5"
server_key_version: "encryptionkey@0"
I20260929 02:23:25.716524 28346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:25.733250 28346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:25.737401 28346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:25.737666 28346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:25.754576 28346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44131
I20260929 02:23:25.754601 28412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44131 every 8 connection(s)
I20260929 02:23:25.755927 28346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:25.759562 28413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:25.765426 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28346
I20260929 02:23:25.765623 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:25.766168 7848 external_mini_cluster.cc:1428] Setting key c9aca1bd7621ad23df96949dbcac65b7
I20260929 02:23:25.770406 28413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: Bootstrap starting.
I20260929 02:23:25.773681 28413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:25.774873 28413 log.cc:824] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:25.777906 28413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: No bootstrap required, opened a new log
I20260929 02:23:25.779201 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:25.767916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43763 (local address 127.0.0.1:44131)
0929 02:23:25.768713 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:25.768725 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:25.768769 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:23:25.770447 (+ 1678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:25.770472 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:25.770882 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:25.771183 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:25.771703 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:25.773049 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:25.774834 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:25.775732 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:25.775933 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:25.777017 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:25.777025 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:25.777040 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:25.777567 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:25.777638 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:25.777664 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:25.777787 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:25.778085 (+ 298us) server_negotiation.cc:299] Negotiation successful
0929 02:23:25.778264 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":372,"threads_started":1}
I20260929 02:23:25.782847 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:25.786716 28413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } }
I20260929 02:23:25.787176 28413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:25.787288 28413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a746009f4fa484bb7395b990553d9b5, State: Initialized, Role: FOLLOWER
I20260929 02:23:25.787911 28413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } }
I20260929 02:23:25.788236 28413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:25.788373 28413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:25.788540 28413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:25.790822 28413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } }
I20260929 02:23:25.791316 28413 leader_election.cc:302] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a746009f4fa484bb7395b990553d9b5; no voters:
I20260929 02:23:25.791976 28413 leader_election.cc:288] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:25.792074 28423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:25.793143 28423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [term 1 LEADER]: Becoming Leader. State: Replica: 1a746009f4fa484bb7395b990553d9b5, State: Running, Role: LEADER
I20260929 02:23:25.793783 28423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } }
I20260929 02:23:25.794271 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:25.787983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36219 (local address 127.0.0.1:44131)
0929 02:23:25.788165 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:25.788170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:25.788540 (+ 370us) server_negotiation.cc:407] Connection header received
0929 02:23:25.788735 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:25.788740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:25.788807 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:25.788894 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:25.789419 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:25.790238 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:25.791649 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:25.792137 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:25.792256 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:25.793560 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:25.793565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:25.793568 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:25.793585 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:25.793621 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:25.793636 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:25.793725 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:25.793917 (+ 192us) server_negotiation.cc:299] Negotiation successful
0929 02:23:25.793970 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:25.794404 28413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:25.796701 28424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a746009f4fa484bb7395b990553d9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } } }
I20260929 02:23:25.797188 28424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:25.797163 28425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a746009f4fa484bb7395b990553d9b5. Latest consensus state: current_term: 1 leader_uuid: "1a746009f4fa484bb7395b990553d9b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a746009f4fa484bb7395b990553d9b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44131 } } }
I20260929 02:23:25.797582 28425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5 [sys.catalog]: This master's current role is: LEADER
W20260929 02:23:25.804586 28439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:23:25.804692 28439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:23:25.804800 28431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:25.806372 28431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:25.814524 28431 catalog_manager.cc:1347] Generated new cluster ID: 93e73c21917547eead39babbc44d0356
I20260929 02:23:25.814630 28431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:25.843629 28431 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:25.844971 28431 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:25.850426 28431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a746009f4fa484bb7395b990553d9b5: Generated new TSK 0
I20260929 02:23:25.851174 28431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:25.857961 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28346
W20260929 02:23:25.859059 28408 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43763
W20260929 02:23:25.859731 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:25.859894 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28346
W20260929 02:23:25.873972 28331 connection.cc:535] client connection to 127.0.0.1:44131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44131): Connection reset by peer (error 104)
2026-09-29T02:23:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1801 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:25 dist-test-slave-8cs2 krb5kdc[28444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:25 dist-test-slave-8cs2 krb5kdc[28444](info): set up 2 sockets
Sep 29 02:23:25 dist-test-slave-8cs2 krb5kdc[28444](info): commencing operation
krb5kdc: starting...
W20260929 02:23:26.893337 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.007s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:27 dist-test-slave-8cs2 krb5kdc[28444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:27.140162 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:27.453635 28457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:27.453922 28457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:27.454018 28457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:27.461807 28457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:27.461917 28457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:27.461967 28457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:27.462026 28457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:27.462066 28457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:27.471951 28457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:27.473225 28457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:27.474499 28457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:27.485863 28462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:27.485814 28465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:27.485829 28463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:27.486672 28457 server_base.cc:1034] running on GCE node
I20260929 02:23:27.487391 28457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:27.488714 28457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:27.489902 28457 hybrid_clock.cc:648] HybridClock initialized: now 1790648607489837 us; error 50 us; skew 500 ppm
I20260929 02:23:27.490202 28457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:27 dist-test-slave-8cs2 krb5kdc[28444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648607, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:27.501415 28457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:27.503024 28457 webserver.cc:460] Webserver started at http://127.0.0.1:45249/ using document root <none> and password file <none>
I20260929 02:23:27.503719 28457 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:27.503831 28457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:27.504098 28457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:27.507149 28457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "14d864ce31334fa383cbdae2fbab04c1"
format_stamp: "Formatted at 2026-09-29 02:23:27 on dist-test-slave-8cs2"
server_key: "b6409da704a1320275c6ad9bf4ab334c"
server_key_iv: "feaae318cd4eccb9defe8c767e9db711"
server_key_version: "encryptionkey@0"
I20260929 02:23:27.507887 28457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "14d864ce31334fa383cbdae2fbab04c1"
format_stamp: "Formatted at 2026-09-29 02:23:27 on dist-test-slave-8cs2"
server_key: "b6409da704a1320275c6ad9bf4ab334c"
server_key_iv: "feaae318cd4eccb9defe8c767e9db711"
server_key_version: "encryptionkey@0"
I20260929 02:23:27.513077 28457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260929 02:23:27.516503 28472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:27.517889 28457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:27.518038 28457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "14d864ce31334fa383cbdae2fbab04c1"
format_stamp: "Formatted at 2026-09-29 02:23:27 on dist-test-slave-8cs2"
server_key: "b6409da704a1320275c6ad9bf4ab334c"
server_key_iv: "feaae318cd4eccb9defe8c767e9db711"
server_key_version: "encryptionkey@0"
I20260929 02:23:27.518200 28457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:27.523579 28457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:27.527570 28457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:27.527797 28457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:27.544740 28457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44087
I20260929 02:23:27.544775 28523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44087 every 8 connection(s)
I20260929 02:23:27.546231 28457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:27.549863 28524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:27.552532 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28457
I20260929 02:23:27.552716 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:27.553252 7848 external_mini_cluster.cc:1428] Setting key 9c6ab78d2e8b18285fec87b1de811966
I20260929 02:23:27.560693 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:27.555029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59289 (local address 127.0.0.1:44087)
0929 02:23:27.555850 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:27.555862 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:27.555904 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:23:27.557545 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:27.557569 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:27.557918 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:27.558226 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:27.559219 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:27.559225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:27.559240 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:27.559262 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:27.559324 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:27.559349 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:27.559514 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:27.559610 (+ 96us) server_negotiation.cc:299] Negotiation successful
0929 02:23:27.559786 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":322,"threads_started":1}
I20260929 02:23:27.561347 28524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1: Bootstrap starting.
I20260929 02:23:27.563957 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:27.564414 28524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:27.565622 28524 log.cc:824] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:27.568583 28524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1: No bootstrap required, opened a new log
I20260929 02:23:27.570777 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:27.568558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45645 (local address 127.0.0.1:44087)
0929 02:23:27.568722 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:27.568727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:27.569177 (+ 450us) server_negotiation.cc:407] Connection header received
0929 02:23:27.569287 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:27.569292 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:27.569348 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:27.569448 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:27.570212 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:27.570217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:27.570221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:27.570242 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:27.570289 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:27.570312 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:27.570422 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:27.570465 (+ 43us) server_negotiation.cc:299] Negotiation successful
0929 02:23:27.570512 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:27.575901 28524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260929 02:23:27.576210 28524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:27.576294 28524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14d864ce31334fa383cbdae2fbab04c1, State: Initialized, Role: FOLLOWER
I20260929 02:23:27.576807 28524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260929 02:23:27.577039 28524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:27.577162 28524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:27.577303 28524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:27.579310 28524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260929 02:23:27.579763 28524 leader_election.cc:302] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14d864ce31334fa383cbdae2fbab04c1; no voters:
I20260929 02:23:27.580309 28524 leader_election.cc:288] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:27.580446 28534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:27.581071 28534 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [term 1 LEADER]: Becoming Leader. State: Replica: 14d864ce31334fa383cbdae2fbab04c1, State: Running, Role: LEADER
I20260929 02:23:27.581791 28534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } }
I20260929 02:23:27.582748 28524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:27.584300 28536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14d864ce31334fa383cbdae2fbab04c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } } }
I20260929 02:23:27.584726 28536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:27.584544 28535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14d864ce31334fa383cbdae2fbab04c1. Latest consensus state: current_term: 1 leader_uuid: "14d864ce31334fa383cbdae2fbab04c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14d864ce31334fa383cbdae2fbab04c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44087 } } }
I20260929 02:23:27.584983 28535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:27.586928 28541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:27.590857 28541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:27.598413 28541 catalog_manager.cc:1347] Generated new cluster ID: 4dbbb696012e45cc9a608de8cd6d2300
I20260929 02:23:27.598515 28541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:27.612516 28541 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:27.613018 28541 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:27.624040 28541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14d864ce31334fa383cbdae2fbab04c1: Generated new TSK 0
I20260929 02:23:27.624878 28541 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:27.638830 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28457
W20260929 02:23:27.640017 28519 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59289
W20260929 02:23:27.640625 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28457: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:27.640744 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28457
2026-09-29T02:23:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1780 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:27 dist-test-slave-8cs2 krb5kdc[28555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:27 dist-test-slave-8cs2 krb5kdc[28555](info): set up 2 sockets
Sep 29 02:23:27 dist-test-slave-8cs2 krb5kdc[28555](info): commencing operation
krb5kdc: starting...
W20260929 02:23:28.649266 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.976s user 0.006s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:28 dist-test-slave-8cs2 krb5kdc[28555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:28.902424 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35417
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:29.223479 28568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:29.223766 28568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:29.223902 28568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:29.231050 28568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:29.231166 28568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:29.231240 28568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:29.231323 28568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:29.231381 28568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:29.240873 28568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:29.242076 28568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:29.243286 28568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:29.254107 28573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:29.254156 28576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:29.254177 28574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:29.255139 28568 server_base.cc:1034] running on GCE node
I20260929 02:23:29.255882 28568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:29.257273 28568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:29.258482 28568 hybrid_clock.cc:648] HybridClock initialized: now 1790648609258433 us; error 43 us; skew 500 ppm
I20260929 02:23:29.258800 28568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:29 dist-test-slave-8cs2 krb5kdc[28555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648609, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:29.261734 28568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:29.263087 28568 webserver.cc:460] Webserver started at http://127.0.0.1:40399/ using document root <none> and password file <none>
I20260929 02:23:29.263739 28568 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:29.263856 28568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:29.264129 28568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:29.266994 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "972b037a6e114f41b6f03cd86cff24dc"
format_stamp: "Formatted at 2026-09-29 02:23:29 on dist-test-slave-8cs2"
server_key: "bd3c480d1e1f30cd2fa57083c97f318b"
server_key_iv: "396eb65372245cbab5374aa7184eb87e"
server_key_version: "encryptionkey@0"
I20260929 02:23:29.267709 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "972b037a6e114f41b6f03cd86cff24dc"
format_stamp: "Formatted at 2026-09-29 02:23:29 on dist-test-slave-8cs2"
server_key: "bd3c480d1e1f30cd2fa57083c97f318b"
server_key_iv: "396eb65372245cbab5374aa7184eb87e"
server_key_version: "encryptionkey@0"
I20260929 02:23:29.272660 28568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:23:29.276243 28583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:29.277592 28568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:29.277758 28568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "972b037a6e114f41b6f03cd86cff24dc"
format_stamp: "Formatted at 2026-09-29 02:23:29 on dist-test-slave-8cs2"
server_key: "bd3c480d1e1f30cd2fa57083c97f318b"
server_key_iv: "396eb65372245cbab5374aa7184eb87e"
server_key_version: "encryptionkey@0"
I20260929 02:23:29.277957 28568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:29.298717 28568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:29.302814 28568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:29.303081 28568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:29.319039 28568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35417
I20260929 02:23:29.319058 28634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35417 every 8 connection(s)
I20260929 02:23:29.320338 28568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:29.323992 28635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:29.328477 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28568
I20260929 02:23:29.328639 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:29.329118 7848 external_mini_cluster.cc:1428] Setting key 9716622734351ae7058f5aa9e3551ba1
I20260929 02:23:29.334488 28635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc: Bootstrap starting.
I20260929 02:23:29.337954 28635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:29.339131 28635 log.cc:824] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:29.342200 28635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc: No bootstrap required, opened a new log
Sep 29 02:23:29 dist-test-slave-8cs2 krb5kdc[28555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:23:29.350857 28635 raft_consensus.cc:348] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260929 02:23:29.351238 28635 raft_consensus.cc:374] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:29.351333 28635 raft_consensus.cc:729] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972b037a6e114f41b6f03cd86cff24dc, State: Initialized, Role: FOLLOWER
I20260929 02:23:29.351578 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:29.330725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59817 (local address 127.0.0.1:35417)
0929 02:23:29.331483 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:29.331494 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:29.331533 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:23:29.333287 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:29.333309 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:29.333695 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:29.333997 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:29.335718 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:29.337197 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:29.338905 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:29.339596 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:29.339765 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:29.344351 (+ 4586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:29.344362 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:29.344375 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:29.344413 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:29.347526 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:29.348294 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:29.348300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:29.348306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:29.348468 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:29.348871 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:29.348874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:29.348876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:29.349449 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:29.349667 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:29.350132 (+ 465us) server_negotiation.cc:299] Negotiation successful
0929 02:23:29.350436 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":346,"threads_started":1}
I20260929 02:23:29.351955 28635 consensus_queue.cc:260] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260929 02:23:29.352200 28635 raft_consensus.cc:388] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:29.352326 28635 raft_consensus.cc:482] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:29.352485 28635 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:29.355381 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:29.355445 28635 raft_consensus.cc:504] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260929 02:23:29.355911 28635 leader_election.cc:302] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 972b037a6e114f41b6f03cd86cff24dc; no voters:
I20260929 02:23:29.356448 28635 leader_election.cc:288] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:29.356580 28640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:29.358219 28635 sys_catalog.cc:564] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:29.365728 28640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [term 1 LEADER]: Becoming Leader. State: Replica: 972b037a6e114f41b6f03cd86cff24dc, State: Running, Role: LEADER
I20260929 02:23:29.366503 28640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } }
I20260929 02:23:29.369797 28642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 972b037a6e114f41b6f03cd86cff24dc. Latest consensus state: current_term: 1 leader_uuid: "972b037a6e114f41b6f03cd86cff24dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260929 02:23:29.370153 28642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:29.370682 28641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "972b037a6e114f41b6f03cd86cff24dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "972b037a6e114f41b6f03cd86cff24dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35417 } } }
I20260929 02:23:29.370945 28641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:29.380055 28656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:29.385426 28638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:29.367539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49077 (local address 127.0.0.1:35417)
0929 02:23:29.367752 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:29.367758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:29.368283 (+ 525us) server_negotiation.cc:407] Connection header received
0929 02:23:29.368563 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:29.368569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:29.368658 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:29.368727 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:29.370809 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:29.371979 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:29.374014 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:29.374758 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:29.374874 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:29.378995 (+ 4121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:29.379003 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:29.379009 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:29.379052 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:29.382687 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:29.383480 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:29.383485 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:29.383490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:29.383615 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:29.383995 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:29.384000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:29.384003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:29.384337 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:29.384565 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:29.384869 (+ 304us) server_negotiation.cc:299] Negotiation successful
0929 02:23:29.385090 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260929 02:23:29.392077 28656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:29.400676 28656 catalog_manager.cc:1347] Generated new cluster ID: 860cf7e448fc4769a0517a319d6d630a
I20260929 02:23:29.400775 28656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:29.425472 28656 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:29.427304 28656 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:29.437049 28656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 972b037a6e114f41b6f03cd86cff24dc: Generated new TSK 0
I20260929 02:23:29.438103 28656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:29.445544 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28568
I20260929 02:23:29.446533 28630 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:29.537063 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28568
2026-09-29T02:23:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1896 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:29 dist-test-slave-8cs2 krb5kdc[28668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:29 dist-test-slave-8cs2 krb5kdc[28668](info): set up 2 sockets
Sep 29 02:23:29 dist-test-slave-8cs2 krb5kdc[28668](info): commencing operation
krb5kdc: starting...
W20260929 02:23:30.545302 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.975s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:30 dist-test-slave-8cs2 krb5kdc[28668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:30.800102 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43389
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:31.118757 28681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:31.119037 28681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:31.119166 28681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:31.126363 28681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:31.126477 28681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:31.126545 28681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:31.126624 28681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:31.126685 28681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:31.136622 28681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:31.137981 28681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:31.139305 28681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:31.151012 28686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:31.150975 28687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:31.150942 28689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:31.151640 28681 server_base.cc:1034] running on GCE node
I20260929 02:23:31.152587 28681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:31.154073 28681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:31.155284 28681 hybrid_clock.cc:648] HybridClock initialized: now 1790648611155243 us; error 31 us; skew 500 ppm
I20260929 02:23:31.155610 28681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:31 dist-test-slave-8cs2 krb5kdc[28668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648611, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:31.158628 28681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:31.160082 28681 webserver.cc:460] Webserver started at http://127.0.0.1:42207/ using document root <none> and password file <none>
I20260929 02:23:31.160728 28681 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:31.160846 28681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:31.161170 28681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:31.164073 28681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "4d552c5eef514a9498d152f7f7dbe3b5"
format_stamp: "Formatted at 2026-09-29 02:23:31 on dist-test-slave-8cs2"
server_key: "7519ce61843fc8fc92bcf947315b546e"
server_key_iv: "440573fe17856281e9a8cb12500267f5"
server_key_version: "encryptionkey@0"
I20260929 02:23:31.164783 28681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "4d552c5eef514a9498d152f7f7dbe3b5"
format_stamp: "Formatted at 2026-09-29 02:23:31 on dist-test-slave-8cs2"
server_key: "7519ce61843fc8fc92bcf947315b546e"
server_key_iv: "440573fe17856281e9a8cb12500267f5"
server_key_version: "encryptionkey@0"
I20260929 02:23:31.169663 28681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260929 02:23:31.173121 28696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:31.174456 28681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260929 02:23:31.174611 28681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "4d552c5eef514a9498d152f7f7dbe3b5"
format_stamp: "Formatted at 2026-09-29 02:23:31 on dist-test-slave-8cs2"
server_key: "7519ce61843fc8fc92bcf947315b546e"
server_key_iv: "440573fe17856281e9a8cb12500267f5"
server_key_version: "encryptionkey@0"
I20260929 02:23:31.174786 28681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:31.195338 28681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:31.199723 28681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:31.199994 28681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:31.217248 28681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43389
I20260929 02:23:31.217257 28747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43389 every 8 connection(s)
I20260929 02:23:31.218577 28681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:31.222240 28748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:31.223860 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28681
I20260929 02:23:31.224068 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:31.224607 7848 external_mini_cluster.cc:1428] Setting key 5f33e44bae15e2d6b896d36d1b717e44
I20260929 02:23:31.236047 28748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5: Bootstrap starting.
I20260929 02:23:31.238046 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:31.226580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39799 (local address 127.0.0.1:43389)
0929 02:23:31.227376 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:31.227387 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:31.227428 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:23:31.229096 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:31.229123 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:31.229541 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:31.229858 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:31.230402 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:31.231682 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:31.233635 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:31.234344 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:31.234481 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:31.235705 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:31.235712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:31.235727 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:31.236310 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:31.236389 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:31.236414 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:31.236501 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:31.236731 (+ 230us) server_negotiation.cc:299] Negotiation successful
0929 02:23:31.237031 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":345,"threads_started":1}
I20260929 02:23:31.239380 28748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:31.240443 28748 log.cc:824] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:31.241600 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:31.243253 28748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5: No bootstrap required, opened a new log
I20260929 02:23:31.252614 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:31.245908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57951 (local address 127.0.0.1:43389)
0929 02:23:31.246110 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:31.246120 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:31.246407 (+ 287us) server_negotiation.cc:407] Connection header received
0929 02:23:31.246629 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:31.246633 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:31.246700 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:31.246786 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:31.247386 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:31.248148 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:31.250040 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:31.250579 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:31.250680 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:31.251886 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:31.251891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:31.251894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:31.251912 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:31.251951 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:31.251965 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:31.252040 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:31.252222 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 02:23:31.252300 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260929 02:23:31.252620 28748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } }
I20260929 02:23:31.253216 28748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:31.253317 28748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d552c5eef514a9498d152f7f7dbe3b5, State: Initialized, Role: FOLLOWER
I20260929 02:23:31.254001 28748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } }
I20260929 02:23:31.254242 28748 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:31.254377 28748 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:31.254530 28748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:31.256657 28748 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } }
I20260929 02:23:31.257186 28748 leader_election.cc:302] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d552c5eef514a9498d152f7f7dbe3b5; no voters:
I20260929 02:23:31.257752 28748 leader_election.cc:288] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:31.257918 28758 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:31.258975 28758 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [term 1 LEADER]: Becoming Leader. State: Replica: 4d552c5eef514a9498d152f7f7dbe3b5, State: Running, Role: LEADER
I20260929 02:23:31.259588 28758 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } }
I20260929 02:23:31.259866 28748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:31.262779 28760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d552c5eef514a9498d152f7f7dbe3b5. Latest consensus state: current_term: 1 leader_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } } }
I20260929 02:23:31.262830 28759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d552c5eef514a9498d152f7f7dbe3b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43389 } } }
I20260929 02:23:31.263197 28759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:31.263278 28760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:31.265019 28767 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:31.269502 28767 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:31.277878 28767 catalog_manager.cc:1347] Generated new cluster ID: 8bd33876df384f5e90ac42b315d6c30e
I20260929 02:23:31.277999 28767 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:31.297660 28767 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:31.299074 28767 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:31.308197 28767 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d552c5eef514a9498d152f7f7dbe3b5: Generated new TSK 0
I20260929 02:23:31.309103 28767 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:31.327085 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28681
W20260929 02:23:31.328125 28743 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39799
W20260929 02:23:31.328820 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28681: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:31.328991 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28681
2026-09-29T02:23:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1793 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:31 dist-test-slave-8cs2 krb5kdc[28779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:31 dist-test-slave-8cs2 krb5kdc[28779](info): set up 2 sockets
Sep 29 02:23:31 dist-test-slave-8cs2 krb5kdc[28779](info): commencing operation
krb5kdc: starting...
W20260929 02:23:32.345360 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.982s user 0.007s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:32 dist-test-slave-8cs2 krb5kdc[28779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648612, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:32.598155 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33997
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33997
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:32.913954 28792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:32.914247 28792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:32.914343 28792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:32.921545 28792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:32.921651 28792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:32.921710 28792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:32.921769 28792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:32.921813 28792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:32.931476 28792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:32.932652 28792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:32.933949 28792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:32.945235 28797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:32.945289 28798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:32.945211 28800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:32.946326 28792 server_base.cc:1034] running on GCE node
I20260929 02:23:32.947046 28792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:32.948361 28792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:32.949555 28792 hybrid_clock.cc:648] HybridClock initialized: now 1790648612949511 us; error 40 us; skew 500 ppm
I20260929 02:23:32.949873 28792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:32 dist-test-slave-8cs2 krb5kdc[28779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648612, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:32.958343 28792 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:32.959741 28792 webserver.cc:460] Webserver started at http://127.0.0.1:39893/ using document root <none> and password file <none>
I20260929 02:23:32.960392 28792 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:32.960505 28792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:32.960772 28792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:32.963653 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "4d59833c93ad4a8688f9a489c6d12c82"
format_stamp: "Formatted at 2026-09-29 02:23:32 on dist-test-slave-8cs2"
server_key: "83e5812b29da426d7fb0da9f43012304"
server_key_iv: "3a1cf8fd33f83c586a12c1144190d3aa"
server_key_version: "encryptionkey@0"
I20260929 02:23:32.964383 28792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "4d59833c93ad4a8688f9a489c6d12c82"
format_stamp: "Formatted at 2026-09-29 02:23:32 on dist-test-slave-8cs2"
server_key: "83e5812b29da426d7fb0da9f43012304"
server_key_iv: "3a1cf8fd33f83c586a12c1144190d3aa"
server_key_version: "encryptionkey@0"
I20260929 02:23:32.969465 28792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260929 02:23:32.972775 28807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:32.974077 28792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:32.974221 28792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "4d59833c93ad4a8688f9a489c6d12c82"
format_stamp: "Formatted at 2026-09-29 02:23:32 on dist-test-slave-8cs2"
server_key: "83e5812b29da426d7fb0da9f43012304"
server_key_iv: "3a1cf8fd33f83c586a12c1144190d3aa"
server_key_version: "encryptionkey@0"
I20260929 02:23:32.974390 28792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:32.979771 28792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:32.983729 28792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:32.983989 28792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:33.000551 28792 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33997
I20260929 02:23:33.000573 28858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33997 every 8 connection(s)
I20260929 02:23:33.001965 28792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:33.005647 28859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:33.011184 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28792
I20260929 02:23:33.011339 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:33.011739 7848 external_mini_cluster.cc:1428] Setting key a9cfab0103f06847559af0b5692b092e
I20260929 02:23:33.016233 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82: Bootstrap starting.
I20260929 02:23:33.019266 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:33.013487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32967 (local address 127.0.0.1:33997)
0929 02:23:33.014314 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:33.014327 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:33.014366 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:23:33.015980 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:33.016003 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:33.016349 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:33.016714 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:33.017739 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:33.017749 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:33.017761 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:33.017775 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:33.017835 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:33.017855 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:33.018058 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:33.018190 (+ 132us) server_negotiation.cc:299] Negotiation successful
0929 02:23:33.018367 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":377,"threads_started":1}
I20260929 02:23:33.019498 28859 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:33.020768 28859 log.cc:824] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:33.023132 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:33.023865 28859 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82: No bootstrap required, opened a new log
I20260929 02:23:33.030447 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:33.028172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38859 (local address 127.0.0.1:33997)
0929 02:23:33.028350 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:33.028356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:33.028635 (+ 279us) server_negotiation.cc:407] Connection header received
0929 02:23:33.028803 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:33.028810 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:33.028867 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:33.028971 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:33.029816 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:33.029822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:33.029827 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:33.029847 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:33.029889 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:33.029906 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:33.030031 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:33.030112 (+ 81us) server_negotiation.cc:299] Negotiation successful
0929 02:23:33.030176 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260929 02:23:33.032878 28859 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } }
I20260929 02:23:33.033325 28859 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:33.033422 28859 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d59833c93ad4a8688f9a489c6d12c82, State: Initialized, Role: FOLLOWER
I20260929 02:23:33.034050 28859 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } }
I20260929 02:23:33.034272 28859 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:33.034402 28859 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:33.034559 28859 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:33.037067 28859 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } }
I20260929 02:23:33.037576 28859 leader_election.cc:302] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d59833c93ad4a8688f9a489c6d12c82; no voters:
I20260929 02:23:33.038129 28859 leader_election.cc:288] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:33.038256 28869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:33.038862 28869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [term 1 LEADER]: Becoming Leader. State: Replica: 4d59833c93ad4a8688f9a489c6d12c82, State: Running, Role: LEADER
I20260929 02:23:33.039448 28869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } }
I20260929 02:23:33.040251 28859 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:33.042332 28871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d59833c93ad4a8688f9a489c6d12c82. Latest consensus state: current_term: 1 leader_uuid: "4d59833c93ad4a8688f9a489c6d12c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } } }
I20260929 02:23:33.042917 28871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:33.044209 28877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:33.044201 28870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d59833c93ad4a8688f9a489c6d12c82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d59833c93ad4a8688f9a489c6d12c82" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33997 } } }
I20260929 02:23:33.044667 28870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:33.049685 28877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:33.059510 28877 catalog_manager.cc:1347] Generated new cluster ID: 89582b2873a542cb99143bc11e6ef122
I20260929 02:23:33.059634 28877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:33.068559 28877 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:33.068995 28877 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:33.074498 28877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d59833c93ad4a8688f9a489c6d12c82: Generated new TSK 0
I20260929 02:23:33.075276 28877 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:33.088997 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28792
W20260929 02:23:33.090044 28854 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:32967
W20260929 02:23:33.090731 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:33.090850 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28792
2026-09-29T02:23:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1759 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:33 dist-test-slave-8cs2 krb5kdc[28890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:33 dist-test-slave-8cs2 krb5kdc[28890](info): set up 2 sockets
Sep 29 02:23:33 dist-test-slave-8cs2 krb5kdc[28890](info): commencing operation
krb5kdc: starting...
W20260929 02:23:34.097333 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.973s user 0.008s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:34 dist-test-slave-8cs2 krb5kdc[28890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:34.359491 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:34.678632 28903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:34.679049 28903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:34.679176 28903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:34.686816 28903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:34.686967 28903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:34.687029 28903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:34.687121 28903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:34.687176 28903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:34.697463 28903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39767
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:34.698817 28903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:34.700129 28903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:34.711555 28908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:34.711607 28909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:34.711629 28911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:34.712566 28903 server_base.cc:1034] running on GCE node
I20260929 02:23:34.713388 28903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:34.714841 28903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:34.716048 28903 hybrid_clock.cc:648] HybridClock initialized: now 1790648614716004 us; error 39 us; skew 500 ppm
I20260929 02:23:34.716387 28903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:34 dist-test-slave-8cs2 krb5kdc[28890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648614, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:34.719488 28903 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:34.721086 28903 webserver.cc:460] Webserver started at http://127.0.0.1:46565/ using document root <none> and password file <none>
I20260929 02:23:34.721779 28903 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:34.721894 28903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:34.722195 28903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:34.725237 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "8ae0cfdfda5049d9bf0358e124691b06"
format_stamp: "Formatted at 2026-09-29 02:23:34 on dist-test-slave-8cs2"
server_key: "928a569e460a7d609dddc2454e6eaa2b"
server_key_iv: "4840fe8f05bc2a5876a01886b2dbfb3d"
server_key_version: "encryptionkey@0"
I20260929 02:23:34.725993 28903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "8ae0cfdfda5049d9bf0358e124691b06"
format_stamp: "Formatted at 2026-09-29 02:23:34 on dist-test-slave-8cs2"
server_key: "928a569e460a7d609dddc2454e6eaa2b"
server_key_iv: "4840fe8f05bc2a5876a01886b2dbfb3d"
server_key_version: "encryptionkey@0"
I20260929 02:23:34.730962 28903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:34.735330 28918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:34.737007 28903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260929 02:23:34.737193 28903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "8ae0cfdfda5049d9bf0358e124691b06"
format_stamp: "Formatted at 2026-09-29 02:23:34 on dist-test-slave-8cs2"
server_key: "928a569e460a7d609dddc2454e6eaa2b"
server_key_iv: "4840fe8f05bc2a5876a01886b2dbfb3d"
server_key_version: "encryptionkey@0"
I20260929 02:23:34.737423 28903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:34.749286 28903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:34.752751 28903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:34.753041 28903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:34.768586 28903 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39767
I20260929 02:23:34.768616 28969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39767 every 8 connection(s)
I20260929 02:23:34.769894 28903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:34.773540 28970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:34.773861 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 28903
I20260929 02:23:34.774019 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:34.774439 7848 external_mini_cluster.cc:1428] Setting key b8a07cb46c20574ab7f7e86f64448001
I20260929 02:23:34.787056 28970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06: Bootstrap starting.
Sep 29 02:23:34 dist-test-slave-8cs2 krb5kdc[28890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:23:34.790058 28970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:34.791147 28970 log.cc:824] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:34.793887 28970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06: No bootstrap required, opened a new log
I20260929 02:23:34.798025 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:34.776167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:52785 (local address 127.0.0.1:39767)
0929 02:23:34.776883 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:34.776897 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:34.776968 (+ 71us) server_negotiation.cc:407] Connection header received
0929 02:23:34.779644 (+ 2676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:34.779673 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:34.780201 (+ 528us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:34.780623 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:34.782288 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:34.784235 (+ 1947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:34.785515 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:34.786484 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:34.786665 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:34.790674 (+ 4009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:34.790682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:34.790696 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:34.790730 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:34.793777 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:34.794706 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:34.794712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:34.794717 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:34.794910 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:34.795361 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:34.795364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:34.795366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:34.795928 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:34.796166 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:34.796650 (+ 484us) server_negotiation.cc:299] Negotiation successful
0929 02:23:34.797039 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":287,"threads_started":1}
I20260929 02:23:34.801987 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:34.802570 28970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } }
I20260929 02:23:34.802891 28970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:34.802975 28970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ae0cfdfda5049d9bf0358e124691b06, State: Initialized, Role: FOLLOWER
I20260929 02:23:34.803486 28970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } }
I20260929 02:23:34.803680 28970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:34.803797 28970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:34.803925 28970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:34.806633 28970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } }
I20260929 02:23:34.807077 28970 leader_election.cc:302] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ae0cfdfda5049d9bf0358e124691b06; no voters:
I20260929 02:23:34.807607 28970 leader_election.cc:288] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:34.807739 28980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:34.809230 28980 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [term 1 LEADER]: Becoming Leader. State: Replica: 8ae0cfdfda5049d9bf0358e124691b06, State: Running, Role: LEADER
I20260929 02:23:34.809686 28970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:34.809808 28980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } }
I20260929 02:23:34.813150 28981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ae0cfdfda5049d9bf0358e124691b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } } }
I20260929 02:23:34.813520 28981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:34.812708 28982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ae0cfdfda5049d9bf0358e124691b06. Latest consensus state: current_term: 1 leader_uuid: "8ae0cfdfda5049d9bf0358e124691b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ae0cfdfda5049d9bf0358e124691b06" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39767 } } }
I20260929 02:23:34.814082 28982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:34.821018 28987 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:34.822507 28973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:34.806210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:32907 (local address 127.0.0.1:39767)
0929 02:23:34.806398 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:34.806404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:34.806877 (+ 473us) server_negotiation.cc:407] Connection header received
0929 02:23:34.807046 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:34.807050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:34.807111 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:34.807185 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:34.808992 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:34.810150 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:34.811800 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:34.812452 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:34.812597 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:34.815951 (+ 3354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:34.815958 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:34.815964 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:34.816006 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:34.819219 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:34.819955 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:34.819962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:34.819964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:34.820082 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:34.820584 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:34.820589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:34.820593 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:34.820915 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:34.821238 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:34.821814 (+ 576us) server_negotiation.cc:299] Negotiation successful
0929 02:23:34.821998 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:34.827052 28987 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:34.836504 28987 catalog_manager.cc:1347] Generated new cluster ID: 0d92e301612c4eed83299cc2136538bd
I20260929 02:23:34.836607 28987 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:34.860483 28987 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:34.862385 28987 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:34.871483 28987 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ae0cfdfda5049d9bf0358e124691b06: Generated new TSK 0
I20260929 02:23:34.872501 28987 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:34.881928 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 28903
I20260929 02:23:34.882961 28965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:34.972995 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 28903
W20260929 02:23:34.987221 28888 connection.cc:535] client connection to 127.0.0.1:39767 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39767): Connection reset by peer (error 104)
2026-09-29T02:23:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1881 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:35 dist-test-slave-8cs2 krb5kdc[29002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:35 dist-test-slave-8cs2 krb5kdc[29002](info): set up 2 sockets
Sep 29 02:23:35 dist-test-slave-8cs2 krb5kdc[29002](info): commencing operation
krb5kdc: starting...
W20260929 02:23:35.997356 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.992s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:36 dist-test-slave-8cs2 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648616, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:36.252557 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46013
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:36.582059 29015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:36.582453 29015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:36.582585 29015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:36.589640 29015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:36.589742 29015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:36.589812 29015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:36.589896 29015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:36.589955 29015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:36.599653 29015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:36.600914 29015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:36.602212 29015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:36.613420 29020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:36.613498 29021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:36.613641 29023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:36.614634 29015 server_base.cc:1034] running on GCE node
I20260929 02:23:36.615449 29015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:36.616900 29015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:36.618140 29015 hybrid_clock.cc:648] HybridClock initialized: now 1790648616618094 us; error 36 us; skew 500 ppm
I20260929 02:23:36.618481 29015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:36 dist-test-slave-8cs2 krb5kdc[29002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:36.621476 29015 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:36.622903 29015 webserver.cc:460] Webserver started at http://127.0.0.1:45465/ using document root <none> and password file <none>
I20260929 02:23:36.623627 29015 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:36.623740 29015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:36.624043 29015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:36.627048 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "a25446575afe41f788c2d99aafc5c02a"
format_stamp: "Formatted at 2026-09-29 02:23:36 on dist-test-slave-8cs2"
server_key: "3cb5acfad9d300cde88afa749c63e170"
server_key_iv: "5f55651df252e955d15af098c7c81352"
server_key_version: "encryptionkey@0"
I20260929 02:23:36.627787 29015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "a25446575afe41f788c2d99aafc5c02a"
format_stamp: "Formatted at 2026-09-29 02:23:36 on dist-test-slave-8cs2"
server_key: "3cb5acfad9d300cde88afa749c63e170"
server_key_iv: "5f55651df252e955d15af098c7c81352"
server_key_version: "encryptionkey@0"
I20260929 02:23:36.633247 29015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260929 02:23:36.636729 29030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:36.638103 29015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:36.638254 29015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "a25446575afe41f788c2d99aafc5c02a"
format_stamp: "Formatted at 2026-09-29 02:23:36 on dist-test-slave-8cs2"
server_key: "3cb5acfad9d300cde88afa749c63e170"
server_key_iv: "5f55651df252e955d15af098c7c81352"
server_key_version: "encryptionkey@0"
I20260929 02:23:36.638437 29015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:36.664120 29015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:36.668030 29015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:36.668272 29015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:36.684422 29015 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46013
I20260929 02:23:36.684450 29081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46013 every 8 connection(s)
I20260929 02:23:36.685755 29015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:36.687354 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29015
I20260929 02:23:36.687528 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:36.687914 7848 external_mini_cluster.cc:1428] Setting key 169f86d0f3f92ae7c2a0d05eb649cb5a
I20260929 02:23:36.689832 29082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:36.702322 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:36.689663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:54355 (local address 127.0.0.1:46013)
0929 02:23:36.690410 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:36.690424 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:36.690483 (+ 59us) server_negotiation.cc:407] Connection header received
0929 02:23:36.692564 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:36.692591 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:36.693129 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:36.693565 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:36.694057 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:36.696078 (+ 2021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:36.697380 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:36.698376 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:36.698557 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:36.699650 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:36.699660 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:36.699678 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:36.700396 (+ 718us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:36.700497 (+ 101us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:36.700531 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:36.700694 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:36.700874 (+ 180us) server_negotiation.cc:299] Negotiation successful
0929 02:23:36.701075 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":313,"threads_started":1}
I20260929 02:23:36.703575 29082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a: Bootstrap starting.
I20260929 02:23:36.705871 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:36.706600 29082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:36.707544 29082 log.cc:824] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:36.710601 29082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a: No bootstrap required, opened a new log
I20260929 02:23:36.715917 29085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:36.709784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:53893 (local address 127.0.0.1:46013)
0929 02:23:36.709915 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:36.709918 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:36.710362 (+ 444us) server_negotiation.cc:407] Connection header received
0929 02:23:36.710514 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:36.710519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:36.710594 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:36.710692 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:36.711161 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:36.712209 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:36.713476 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:36.714137 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:36.714262 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:36.715217 (+ 955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:36.715224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:36.715229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:36.715252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:36.715299 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:36.715319 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:36.715442 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:36.715548 (+ 106us) server_negotiation.cc:299] Negotiation successful
0929 02:23:36.715599 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 02:23:36.719230 29082 raft_consensus.cc:348] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260929 02:23:36.719550 29082 raft_consensus.cc:374] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:36.719619 29082 raft_consensus.cc:729] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a25446575afe41f788c2d99aafc5c02a, State: Initialized, Role: FOLLOWER
I20260929 02:23:36.720124 29082 consensus_queue.cc:260] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260929 02:23:36.720310 29082 raft_consensus.cc:388] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:36.720403 29082 raft_consensus.cc:482] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:36.720534 29082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:36.722886 29082 raft_consensus.cc:504] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260929 02:23:36.723297 29082 leader_election.cc:302] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a25446575afe41f788c2d99aafc5c02a; no voters:
I20260929 02:23:36.723766 29082 leader_election.cc:288] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:36.723882 29092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:36.724490 29092 raft_consensus.cc:686] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [term 1 LEADER]: Becoming Leader. State: Replica: a25446575afe41f788c2d99aafc5c02a, State: Running, Role: LEADER
I20260929 02:23:36.725060 29092 consensus_queue.cc:237] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } }
I20260929 02:23:36.725580 29082 sys_catalog.cc:564] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:36.727543 29094 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a25446575afe41f788c2d99aafc5c02a. Latest consensus state: current_term: 1 leader_uuid: "a25446575afe41f788c2d99aafc5c02a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } } }
I20260929 02:23:36.728029 29094 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:36.728114 29093 sys_catalog.cc:455] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a25446575afe41f788c2d99aafc5c02a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a25446575afe41f788c2d99aafc5c02a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46013 } } }
I20260929 02:23:36.728534 29093 sys_catalog.cc:458] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:36.742725 29098 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:36.747201 29098 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:36.755167 29098 catalog_manager.cc:1347] Generated new cluster ID: 1bd021e18a5644309a727bcf2f90e6ef
I20260929 02:23:36.755271 29098 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:36.777153 29098 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:36.778890 29098 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:36.803381 29098 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a25446575afe41f788c2d99aafc5c02a: Generated new TSK 0
I20260929 02:23:36.804463 29098 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:36.821544 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29015
W20260929 02:23:36.822597 29077 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.168:54355
W20260929 02:23:36.823320 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29015: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:36.823446 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29015
W20260929 02:23:36.837651 29000 connection.cc:535] client connection to 127.0.0.1:46013 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46013): Connection reset by peer (error 104)
2026-09-29T02:23:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1850 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:36 dist-test-slave-8cs2 krb5kdc[29113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:36 dist-test-slave-8cs2 krb5kdc[29113](info): set up 2 sockets
Sep 29 02:23:36 dist-test-slave-8cs2 krb5kdc[29113](info): commencing operation
krb5kdc: starting...
W20260929 02:23:37.845502 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:38 dist-test-slave-8cs2 krb5kdc[29113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:38.102147 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:38.420411 29126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:38.420704 29126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:38.420853 29126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:38.428258 29126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:38.428406 29126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:38.428485 29126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:38.428567 29126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:38.428627 29126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:38.438488 29126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:38.439741 29126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:38.441048 29126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:38.451992 29132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:38.451933 29131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:38.452024 29134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:38.452718 29126 server_base.cc:1034] running on GCE node
I20260929 02:23:38.453502 29126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:38.454861 29126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:38.456116 29126 hybrid_clock.cc:648] HybridClock initialized: now 1790648618456073 us; error 34 us; skew 500 ppm
I20260929 02:23:38.456427 29126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:38 dist-test-slave-8cs2 krb5kdc[29113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648618, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:38.477592 29126 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:38.479115 29126 webserver.cc:460] Webserver started at http://127.0.0.1:34217/ using document root <none> and password file <none>
I20260929 02:23:38.479773 29126 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:38.479919 29126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:38.480202 29126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:38.483253 29126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "8c83ed21cba541938184ef8a3a2abc48"
format_stamp: "Formatted at 2026-09-29 02:23:38 on dist-test-slave-8cs2"
server_key: "62bc284891e0e3f828343b1da6fa7cc4"
server_key_iv: "3dec42e6d71738b17ab561a6a7dc6fc7"
server_key_version: "encryptionkey@0"
I20260929 02:23:38.484014 29126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "8c83ed21cba541938184ef8a3a2abc48"
format_stamp: "Formatted at 2026-09-29 02:23:38 on dist-test-slave-8cs2"
server_key: "62bc284891e0e3f828343b1da6fa7cc4"
server_key_iv: "3dec42e6d71738b17ab561a6a7dc6fc7"
server_key_version: "encryptionkey@0"
I20260929 02:23:38.489334 29126 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260929 02:23:38.492648 29141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:38.493975 29126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:38.494123 29126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "8c83ed21cba541938184ef8a3a2abc48"
format_stamp: "Formatted at 2026-09-29 02:23:38 on dist-test-slave-8cs2"
server_key: "62bc284891e0e3f828343b1da6fa7cc4"
server_key_iv: "3dec42e6d71738b17ab561a6a7dc6fc7"
server_key_version: "encryptionkey@0"
I20260929 02:23:38.494293 29126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:38.499742 29126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:38.503705 29126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:38.503973 29126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:38.520607 29126 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42017
I20260929 02:23:38.520646 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42017 every 8 connection(s)
I20260929 02:23:38.521993 29126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:38.525591 29193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:38.525911 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29126
I20260929 02:23:38.526060 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:38.526503 7848 external_mini_cluster.cc:1428] Setting key 48960262bbcac9d2021e11378cd056ee
I20260929 02:23:38.534493 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:38.528168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:60657 (local address 127.0.0.1:42017)
0929 02:23:38.528997 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:38.529012 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:38.529062 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:23:38.530581 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:38.530610 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:38.531076 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:38.531557 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:38.532531 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:38.532540 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:38.532558 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:38.532581 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:38.532651 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:38.532683 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:38.532899 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:38.533072 (+ 173us) server_negotiation.cc:299] Negotiation successful
0929 02:23:38.533277 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":302,"threads_started":1}
I20260929 02:23:38.538064 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:38.539872 29193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48: Bootstrap starting.
I20260929 02:23:38.543813 29193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:38.544996 29193 log.cc:824] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:38.545416 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:38.543159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:55053 (local address 127.0.0.1:42017)
0929 02:23:38.543329 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:38.543334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:38.543638 (+ 304us) server_negotiation.cc:407] Connection header received
0929 02:23:38.543774 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:38.543779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:38.543846 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:38.543964 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:38.544769 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:38.544775 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:38.544779 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:38.544797 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:38.544841 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:38.544860 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:38.545002 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:38.545093 (+ 91us) server_negotiation.cc:299] Negotiation successful
0929 02:23:38.545153 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260929 02:23:38.548343 29193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48: No bootstrap required, opened a new log
I20260929 02:23:38.555600 29193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } }
I20260929 02:23:38.556030 29193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:38.556141 29193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c83ed21cba541938184ef8a3a2abc48, State: Initialized, Role: FOLLOWER
I20260929 02:23:38.556784 29193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } }
I20260929 02:23:38.557062 29193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:38.557199 29193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:38.557343 29193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:38.559478 29193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } }
I20260929 02:23:38.559967 29193 leader_election.cc:302] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c83ed21cba541938184ef8a3a2abc48; no voters:
I20260929 02:23:38.560509 29193 leader_election.cc:288] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:38.560647 29203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:38.561677 29203 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [term 1 LEADER]: Becoming Leader. State: Replica: 8c83ed21cba541938184ef8a3a2abc48, State: Running, Role: LEADER
I20260929 02:23:38.562263 29203 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } }
I20260929 02:23:38.562711 29193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:38.564697 29205 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c83ed21cba541938184ef8a3a2abc48. Latest consensus state: current_term: 1 leader_uuid: "8c83ed21cba541938184ef8a3a2abc48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } } }
I20260929 02:23:38.565321 29205 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:38.565409 29204 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c83ed21cba541938184ef8a3a2abc48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c83ed21cba541938184ef8a3a2abc48" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42017 } } }
I20260929 02:23:38.565678 29204 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:38.568007 29209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:38.573060 29209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:38.582581 29209 catalog_manager.cc:1347] Generated new cluster ID: 4da8fe78228c4ef080828ac777cdbe1d
I20260929 02:23:38.582701 29209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:38.602991 29209 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:38.603510 29209 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:38.613528 29209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c83ed21cba541938184ef8a3a2abc48: Generated new TSK 0
I20260929 02:23:38.614358 29209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:38.624634 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29126
W20260929 02:23:38.625748 29188 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.168:60657
W20260929 02:23:38.626572 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29126: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:38.626725 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29126
2026-09-29T02:23:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1803 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:38 dist-test-slave-8cs2 krb5kdc[29224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:38 dist-test-slave-8cs2 krb5kdc[29224](info): set up 2 sockets
Sep 29 02:23:38 dist-test-slave-8cs2 krb5kdc[29224](info): commencing operation
krb5kdc: starting...
W20260929 02:23:39.645388 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.986s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:39 dist-test-slave-8cs2 krb5kdc[29224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:39.912588 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38215
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:40.248796 29237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:40.249140 29237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:40.249272 29237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:40.256992 29237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:40.257113 29237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:40.257174 29237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:40.257264 29237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:40.257319 29237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:40.267649 29237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38215
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:40.269048 29237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:40.270401 29237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:40.282366 29242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:40.282406 29245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:40.282318 29243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:40.283278 29237 server_base.cc:1034] running on GCE node
I20260929 02:23:40.284139 29237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:40.285691 29237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:40.286898 29237 hybrid_clock.cc:648] HybridClock initialized: now 1790648620286849 us; error 48 us; skew 500 ppm
I20260929 02:23:40.287281 29237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:40 dist-test-slave-8cs2 krb5kdc[29224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648620, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:40.290688 29237 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:40.292250 29237 webserver.cc:460] Webserver started at http://127.0.0.1:39861/ using document root <none> and password file <none>
I20260929 02:23:40.293025 29237 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:40.293151 29237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:40.293452 29237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:40.296515 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "36d81226bc1a447694bdf8f26b9558d3"
format_stamp: "Formatted at 2026-09-29 02:23:40 on dist-test-slave-8cs2"
server_key: "f88c7a57cf916c44faae5af706a9ce77"
server_key_iv: "010a67c533542b8af876b1296c635a94"
server_key_version: "encryptionkey@0"
I20260929 02:23:40.297330 29237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "36d81226bc1a447694bdf8f26b9558d3"
format_stamp: "Formatted at 2026-09-29 02:23:40 on dist-test-slave-8cs2"
server_key: "f88c7a57cf916c44faae5af706a9ce77"
server_key_iv: "010a67c533542b8af876b1296c635a94"
server_key_version: "encryptionkey@0"
I20260929 02:23:40.302677 29237 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:23:40.306339 29252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:40.307773 29237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:40.307997 29237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "36d81226bc1a447694bdf8f26b9558d3"
format_stamp: "Formatted at 2026-09-29 02:23:40 on dist-test-slave-8cs2"
server_key: "f88c7a57cf916c44faae5af706a9ce77"
server_key_iv: "010a67c533542b8af876b1296c635a94"
server_key_version: "encryptionkey@0"
I20260929 02:23:40.308193 29237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:40.333527 29237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:40.338092 29237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:40.338373 29237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:40.355973 29237 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38215
I20260929 02:23:40.356022 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38215 every 8 connection(s)
I20260929 02:23:40.357468 29237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:40.359840 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29237
I20260929 02:23:40.360010 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:40.360460 7848 external_mini_cluster.cc:1428] Setting key d2a6507de5bb466ed08470dd2c83e45d
I20260929 02:23:40.361575 29304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:23:40 dist-test-slave-8cs2 krb5kdc[29224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260929 02:23:40.375378 29304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3: Bootstrap starting.
I20260929 02:23:40.379446 29304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:40.380703 29304 log.cc:824] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:40.384235 29304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3: No bootstrap required, opened a new log
I20260929 02:23:40.384847 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:40.362235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:50677 (local address 127.0.0.1:38215)
0929 02:23:40.363013 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:40.363027 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:40.363077 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:23:40.365065 (+ 1988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:40.365095 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:40.365656 (+ 561us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:40.366158 (+ 502us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:40.367445 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:40.369487 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:40.370795 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:40.371826 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:40.372032 (+ 206us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:40.376185 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:40.376196 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:40.376216 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:40.376263 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:40.380079 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:40.380952 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:40.380961 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:40.380970 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:40.381177 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:40.381723 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:40.381726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:40.381729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:40.382412 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:40.382707 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:40.383163 (+ 456us) server_negotiation.cc:299] Negotiation successful
0929 02:23:40.383505 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":310,"threads_started":1}
I20260929 02:23:40.388865 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:40.393718 29304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260929 02:23:40.394214 29304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:40.394325 29304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36d81226bc1a447694bdf8f26b9558d3, State: Initialized, Role: FOLLOWER
I20260929 02:23:40.394949 29304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260929 02:23:40.395171 29304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:40.395295 29304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:40.395464 29304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:40.397790 29304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260929 02:23:40.398311 29304 leader_election.cc:302] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36d81226bc1a447694bdf8f26b9558d3; no voters:
I20260929 02:23:40.398945 29304 leader_election.cc:288] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:40.399067 29314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:40.399860 29314 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [term 1 LEADER]: Becoming Leader. State: Replica: 36d81226bc1a447694bdf8f26b9558d3, State: Running, Role: LEADER
I20260929 02:23:40.400501 29314 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } }
I20260929 02:23:40.401284 29304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:40.403574 29316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36d81226bc1a447694bdf8f26b9558d3. Latest consensus state: current_term: 1 leader_uuid: "36d81226bc1a447694bdf8f26b9558d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260929 02:23:40.404250 29315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36d81226bc1a447694bdf8f26b9558d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36d81226bc1a447694bdf8f26b9558d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38215 } } }
I20260929 02:23:40.405077 29316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:40.405175 29315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:40.410624 29307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:40.393930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:40821 (local address 127.0.0.1:38215)
0929 02:23:40.394117 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:40.394123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:40.394479 (+ 356us) server_negotiation.cc:407] Connection header received
0929 02:23:40.394669 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:40.394673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:40.394737 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:40.394819 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:40.396307 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:40.397150 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:40.398423 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:40.398944 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:40.399058 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:40.401830 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:40.401835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:40.401838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:40.401870 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:40.405891 (+ 4021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:40.407092 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:40.407098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:40.407102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:40.407267 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:40.407881 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:40.407886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:40.407890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:40.408289 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:40.409807 (+ 1518us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:40.409993 (+ 186us) server_negotiation.cc:299] Negotiation successful
0929 02:23:40.410193 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260929 02:23:40.414386 29320 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:40.420151 29320 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:40.429387 29320 catalog_manager.cc:1347] Generated new cluster ID: a748a82722cb4c078aadc1cac2333239
I20260929 02:23:40.429574 29320 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:40.445300 29320 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:40.446719 29320 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:40.459064 29320 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36d81226bc1a447694bdf8f26b9558d3: Generated new TSK 0
I20260929 02:23:40.460053 29320 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:40.476702 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29237
I20260929 02:23:40.478077 29299 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:40.570657 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29237
W20260929 02:23:40.585520 29222 connection.cc:535] client connection to 127.0.0.1:38215 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38215): Connection reset by peer (error 104)
2026-09-29T02:23:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1944 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:40 dist-test-slave-8cs2 krb5kdc[29336](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:40 dist-test-slave-8cs2 krb5kdc[29336](info): set up 2 sockets
Sep 29 02:23:40 dist-test-slave-8cs2 krb5kdc[29336](info): commencing operation
krb5kdc: starting...
W20260929 02:23:41.605499 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.001s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:41 dist-test-slave-8cs2 krb5kdc[29336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:41.869668 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:42.189229 29349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:42.189508 29349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:42.189606 29349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:42.196736 29349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:42.196846 29349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:42.196895 29349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:42.196983 29349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:42.197041 29349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:42.206699 29349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41385
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:42.207873 29349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:42.209164 29349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:42.220201 29355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:42.220243 29357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:42.220291 29354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:42.221081 29349 server_base.cc:1034] running on GCE node
I20260929 02:23:42.221783 29349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:42.223155 29349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:42.224364 29349 hybrid_clock.cc:648] HybridClock initialized: now 1790648622224303 us; error 45 us; skew 500 ppm
I20260929 02:23:42.224687 29349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:42 dist-test-slave-8cs2 krb5kdc[29336](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648622, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:42.227660 29349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:42.229017 29349 webserver.cc:460] Webserver started at http://127.0.0.1:44343/ using document root <none> and password file <none>
I20260929 02:23:42.229637 29349 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:42.229749 29349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:42.230029 29349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:42.232890 29349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "463d5b30766a4b7c95052a992efa7fd5"
format_stamp: "Formatted at 2026-09-29 02:23:42 on dist-test-slave-8cs2"
server_key: "96135da4c9ee70cc0be26ebef126e357"
server_key_iv: "c6272980887881ed75b99978751312e8"
server_key_version: "encryptionkey@0"
I20260929 02:23:42.233655 29349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "463d5b30766a4b7c95052a992efa7fd5"
format_stamp: "Formatted at 2026-09-29 02:23:42 on dist-test-slave-8cs2"
server_key: "96135da4c9ee70cc0be26ebef126e357"
server_key_iv: "c6272980887881ed75b99978751312e8"
server_key_version: "encryptionkey@0"
I20260929 02:23:42.238479 29349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260929 02:23:42.241760 29364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:42.243127 29349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:42.243281 29349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "463d5b30766a4b7c95052a992efa7fd5"
format_stamp: "Formatted at 2026-09-29 02:23:42 on dist-test-slave-8cs2"
server_key: "96135da4c9ee70cc0be26ebef126e357"
server_key_iv: "c6272980887881ed75b99978751312e8"
server_key_version: "encryptionkey@0"
I20260929 02:23:42.243481 29349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:42.266706 29349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:42.270627 29349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:42.270885 29349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:42.288394 29349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41385
I20260929 02:23:42.288419 29415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41385 every 8 connection(s)
I20260929 02:23:42.289773 29349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:42.293507 29416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:42.295002 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29349
I20260929 02:23:42.295162 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:42.295569 7848 external_mini_cluster.cc:1428] Setting key bc39778ee3c45ae621c84494db0cc97d
I20260929 02:23:42.306926 29416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5: Bootstrap starting.
I20260929 02:23:42.308414 29419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:42.297295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:40993 (local address 127.0.0.1:41385)
0929 02:23:42.298044 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:42.298055 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:42.298102 (+ 47us) server_negotiation.cc:407] Connection header received
0929 02:23:42.299657 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:42.299679 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:42.300100 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:42.300421 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:42.301059 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:42.302285 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:42.304173 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:42.304853 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:42.305036 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:42.306241 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:42.306248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:42.306262 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:42.306774 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:42.306846 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:42.306870 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:42.306986 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:42.307270 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 02:23:42.307454 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":343,"threads_started":1}
I20260929 02:23:42.310155 29416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:42.311269 29416 log.cc:824] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:42.312083 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:42.313979 29416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5: No bootstrap required, opened a new log
I20260929 02:23:42.322743 29419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:42.316421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:34741 (local address 127.0.0.1:41385)
0929 02:23:42.316580 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:42.316584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:42.316858 (+ 274us) server_negotiation.cc:407] Connection header received
0929 02:23:42.317041 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:42.317045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:42.317111 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:42.317193 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:42.317733 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:42.318486 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:42.320326 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:42.320757 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:42.320851 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:42.322039 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:42.322043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:42.322047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:42.322064 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:42.322113 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:42.322128 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:42.322210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:42.322403 (+ 193us) server_negotiation.cc:299] Negotiation successful
0929 02:23:42.322445 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260929 02:23:42.323279 29416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260929 02:23:42.323678 29416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:42.323773 29416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 463d5b30766a4b7c95052a992efa7fd5, State: Initialized, Role: FOLLOWER
I20260929 02:23:42.324331 29416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260929 02:23:42.324556 29416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:42.324692 29416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:42.324847 29416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:42.327018 29416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260929 02:23:42.327513 29416 leader_election.cc:302] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 463d5b30766a4b7c95052a992efa7fd5; no voters:
I20260929 02:23:42.328145 29416 leader_election.cc:288] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:42.328248 29426 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:42.329182 29426 raft_consensus.cc:686] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [term 1 LEADER]: Becoming Leader. State: Replica: 463d5b30766a4b7c95052a992efa7fd5, State: Running, Role: LEADER
I20260929 02:23:42.329782 29426 consensus_queue.cc:237] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } }
I20260929 02:23:42.330511 29416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:42.332849 29427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "463d5b30766a4b7c95052a992efa7fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260929 02:23:42.332906 29428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 463d5b30766a4b7c95052a992efa7fd5. Latest consensus state: current_term: 1 leader_uuid: "463d5b30766a4b7c95052a992efa7fd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "463d5b30766a4b7c95052a992efa7fd5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41385 } } }
I20260929 02:23:42.333258 29427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:42.333263 29428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:42.337534 29434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:42.341816 29434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:42.349985 29434 catalog_manager.cc:1347] Generated new cluster ID: 634d6dc6e6bb4e8f9572b3fad278d5e4
I20260929 02:23:42.350095 29434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:42.370872 29434 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:42.372748 29434 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:42.380251 29434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 463d5b30766a4b7c95052a992efa7fd5: Generated new TSK 0
I20260929 02:23:42.381301 29434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:42.393870 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29349
W20260929 02:23:42.395432 29411 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.168:40993
W20260929 02:23:42.396130 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29349: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:42.396282 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29349
W20260929 02:23:42.410470 29334 connection.cc:535] client connection to 127.0.0.1:41385 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41385): Connection reset by peer (error 104)
2026-09-29T02:23:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1828 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:42 dist-test-slave-8cs2 krb5kdc[29447](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:42 dist-test-slave-8cs2 krb5kdc[29447](info): set up 2 sockets
Sep 29 02:23:42 dist-test-slave-8cs2 krb5kdc[29447](info): commencing operation
krb5kdc: starting...
W20260929 02:23:43.413448 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.980s user 0.006s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:43 dist-test-slave-8cs2 krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260929 02:23:43.681226 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41999
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:44.003264 29460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:44.003567 29460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:44.003695 29460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:44.010825 29460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:44.010939 29460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:44.011018 29460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:44.011107 29460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:44.011164 29460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:44.020951 29460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41999
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:44.022253 29460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:44.023566 29460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:44.034907 29465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:44.034886 29466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:44.035002 29468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:44.035689 29460 server_base.cc:1034] running on GCE node
I20260929 02:23:44.036468 29460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:44.037861 29460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:44.039047 29460 hybrid_clock.cc:648] HybridClock initialized: now 1790648624038997 us; error 43 us; skew 500 ppm
I20260929 02:23:44.039348 29460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:44 dist-test-slave-8cs2 krb5kdc[29447](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648624, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:44.047971 29460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260929 02:23:44.049530 29460 webserver.cc:460] Webserver started at http://127.0.0.1:46515/ using document root <none> and password file <none>
I20260929 02:23:44.050184 29460 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:44.050284 29460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:44.050547 29460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:44.053574 29460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "797537be869f48e18d8b66846b0bf1fa"
format_stamp: "Formatted at 2026-09-29 02:23:44 on dist-test-slave-8cs2"
server_key: "f8ae0d75f338222e4687df65c57e324a"
server_key_iv: "60620b9b68177a1c198470645ae2ed1e"
server_key_version: "encryptionkey@0"
I20260929 02:23:44.054306 29460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "797537be869f48e18d8b66846b0bf1fa"
format_stamp: "Formatted at 2026-09-29 02:23:44 on dist-test-slave-8cs2"
server_key: "f8ae0d75f338222e4687df65c57e324a"
server_key_iv: "60620b9b68177a1c198470645ae2ed1e"
server_key_version: "encryptionkey@0"
I20260929 02:23:44.059614 29460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:23:44.063297 29475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:44.064752 29460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:44.064913 29460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "797537be869f48e18d8b66846b0bf1fa"
format_stamp: "Formatted at 2026-09-29 02:23:44 on dist-test-slave-8cs2"
server_key: "f8ae0d75f338222e4687df65c57e324a"
server_key_iv: "60620b9b68177a1c198470645ae2ed1e"
server_key_version: "encryptionkey@0"
I20260929 02:23:44.065162 29460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:44.070995 29460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:44.075116 29460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:44.075368 29460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:44.092233 29460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41999
I20260929 02:23:44.092280 29526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41999 every 8 connection(s)
I20260929 02:23:44.093704 29460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:44.096706 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29460
I20260929 02:23:44.096840 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:44.097287 7848 external_mini_cluster.cc:1428] Setting key d284275fd91208046cadf54fef541860
I20260929 02:23:44.097723 29527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:44.105412 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:44.099115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:44549 (local address 127.0.0.1:41999)
0929 02:23:44.099904 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:44.099916 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:44.099957 (+ 41us) server_negotiation.cc:407] Connection header received
0929 02:23:44.102098 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:44.102121 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:44.102463 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:44.102784 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:44.103811 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:44.103818 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:44.103831 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:44.103848 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:44.103941 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:44.103968 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:44.104138 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:44.104240 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 02:23:44.104424 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":342,"threads_started":1}
I20260929 02:23:44.108966 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:44.110337 29527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa: Bootstrap starting.
I20260929 02:23:44.113881 29527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:44.115000 29527 log.cc:824] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:44.116317 29530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:44.113085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.168:51903 (local address 127.0.0.1:41999)
0929 02:23:44.113233 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:44.113237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:44.114481 (+ 1244us) server_negotiation.cc:407] Connection header received
0929 02:23:44.114637 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:44.114640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:44.114688 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:44.114775 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:44.115679 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:44.115685 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:44.115690 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:44.115703 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:44.115743 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:44.115759 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:44.115855 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:44.115984 (+ 129us) server_negotiation.cc:299] Negotiation successful
0929 02:23:44.116043 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:23:44.117823 29527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa: No bootstrap required, opened a new log
I20260929 02:23:44.126080 29527 raft_consensus.cc:348] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } }
I20260929 02:23:44.126451 29527 raft_consensus.cc:374] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:44.126552 29527 raft_consensus.cc:729] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 797537be869f48e18d8b66846b0bf1fa, State: Initialized, Role: FOLLOWER
I20260929 02:23:44.127115 29527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } }
I20260929 02:23:44.127326 29527 raft_consensus.cc:388] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:44.127457 29527 raft_consensus.cc:482] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:44.127606 29527 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:44.129783 29527 raft_consensus.cc:504] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } }
I20260929 02:23:44.130262 29527 leader_election.cc:302] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 797537be869f48e18d8b66846b0bf1fa; no voters:
I20260929 02:23:44.130788 29527 leader_election.cc:288] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:44.130930 29537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:44.131524 29537 raft_consensus.cc:686] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [term 1 LEADER]: Becoming Leader. State: Replica: 797537be869f48e18d8b66846b0bf1fa, State: Running, Role: LEADER
I20260929 02:23:44.132081 29537 consensus_queue.cc:237] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } }
I20260929 02:23:44.132867 29527 sys_catalog.cc:564] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:44.134966 29538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "797537be869f48e18d8b66846b0bf1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } } }
I20260929 02:23:44.135408 29538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:44.135748 29539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 797537be869f48e18d8b66846b0bf1fa. Latest consensus state: current_term: 1 leader_uuid: "797537be869f48e18d8b66846b0bf1fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "797537be869f48e18d8b66846b0bf1fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41999 } } }
I20260929 02:23:44.136485 29539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:44.137338 29545 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:44.142474 29545 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:44.151193 29545 catalog_manager.cc:1347] Generated new cluster ID: 1746ff0f138e4d3e993bd8e97a1be403
I20260929 02:23:44.151294 29545 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:44.166857 29545 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:44.167896 29545 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:44.180857 29545 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 797537be869f48e18d8b66846b0bf1fa: Generated new TSK 0
I20260929 02:23:44.181846 29545 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:44.198676 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29460
W20260929 02:23:44.199656 29522 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.168:44549
W20260929 02:23:44.200318 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29460: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:44.200436 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29460
2026-09-29T02:23:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1799 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:44 dist-test-slave-8cs2 krb5kdc[29558](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:44 dist-test-slave-8cs2 krb5kdc[29558](info): set up 2 sockets
Sep 29 02:23:44 dist-test-slave-8cs2 krb5kdc[29558](info): commencing operation
krb5kdc: starting...
W20260929 02:23:45.225559 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.993s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:45 dist-test-slave-8cs2 krb5kdc[29558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:45.490384 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34201
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:34201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:45.818560 29571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:45.818854 29571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:45.818989 29571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:45.826218 29571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:45.826326 29571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:45.826372 29571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:45.826453 29571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:45.826509 29571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:45.836490 29571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:34201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:34201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:45.837783 29571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:45.839080 29571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:45.850584 29576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:45.850531 29579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:45.850584 29577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:45.851624 29571 server_base.cc:1034] running on GCE node
I20260929 02:23:45.852370 29571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:45.853740 29571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:45.854956 29571 hybrid_clock.cc:648] HybridClock initialized: now 1790648625854904 us; error 35 us; skew 500 ppm
I20260929 02:23:45.855261 29571 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:45 dist-test-slave-8cs2 krb5kdc[29558](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648625, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:45.858196 29571 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:45.859560 29571 webserver.cc:460] Webserver started at http://127.7.170.62:46733/ using document root <none> and password file <none>
I20260929 02:23:45.860198 29571 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:45.860293 29571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:45.860540 29571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:45.863430 29571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "2bfd285021d040cda9a0fb349f196ed6"
format_stamp: "Formatted at 2026-09-29 02:23:45 on dist-test-slave-8cs2"
server_key: "0c2d97b6e51a8c3f49e888a6d3cf62a5"
server_key_iv: "0ff79846c181f54c63ca7ef404c54500"
server_key_version: "encryptionkey@0"
I20260929 02:23:45.864156 29571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "2bfd285021d040cda9a0fb349f196ed6"
format_stamp: "Formatted at 2026-09-29 02:23:45 on dist-test-slave-8cs2"
server_key: "0c2d97b6e51a8c3f49e888a6d3cf62a5"
server_key_iv: "0ff79846c181f54c63ca7ef404c54500"
server_key_version: "encryptionkey@0"
I20260929 02:23:45.869392 29571 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260929 02:23:45.872727 29586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:45.874128 29571 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:45.874269 29571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "2bfd285021d040cda9a0fb349f196ed6"
format_stamp: "Formatted at 2026-09-29 02:23:45 on dist-test-slave-8cs2"
server_key: "0c2d97b6e51a8c3f49e888a6d3cf62a5"
server_key_iv: "0ff79846c181f54c63ca7ef404c54500"
server_key_version: "encryptionkey@0"
I20260929 02:23:45.874447 29571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:45.899101 29571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:45.903473 29571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:45.903731 29571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:45.919442 29571 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:34201
I20260929 02:23:45.919485 29637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:34201 every 8 connection(s)
I20260929 02:23:45.920709 29571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:45.924345 29638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:45.925110 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29571
I20260929 02:23:45.925267 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:45.925686 7848 external_mini_cluster.cc:1428] Setting key 2607bd9ccf30a61563c2a28cf9e5488f
I20260929 02:23:45.937340 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6: Bootstrap starting.
Sep 29 02:23:45 dist-test-slave-8cs2 krb5kdc[29558](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:45.941102 29638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:45.942132 29638 log.cc:824] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:45.945142 29638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6: No bootstrap required, opened a new log
I20260929 02:23:45.948333 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:45.927447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54113 (local address 127.7.170.62:34201)
0929 02:23:45.928176 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:45.928186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:45.928232 (+ 46us) server_negotiation.cc:407] Connection header received
0929 02:23:45.929964 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:45.929991 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:45.930389 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:45.930699 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:45.932439 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:45.933814 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:45.935588 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:45.936291 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:45.936428 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:45.941121 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:45.941129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:45.941144 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:45.941186 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:45.944083 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:45.945005 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:45.945011 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:45.945017 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:45.945187 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:45.945726 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:45.945729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:45.945731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:45.946238 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:45.946474 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:45.946992 (+ 518us) server_negotiation.cc:299] Negotiation successful
0929 02:23:45.947275 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":330,"threads_started":1}
I20260929 02:23:45.951972 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:45.953234 29638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } }
I20260929 02:23:45.953557 29638 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:45.953648 29638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bfd285021d040cda9a0fb349f196ed6, State: Initialized, Role: FOLLOWER
I20260929 02:23:45.954160 29638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } }
I20260929 02:23:45.954386 29638 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:45.954511 29638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:45.954670 29638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:45.957309 29638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } }
I20260929 02:23:45.957953 29638 leader_election.cc:302] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bfd285021d040cda9a0fb349f196ed6; no voters:
I20260929 02:23:45.958571 29638 leader_election.cc:288] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:45.958635 29648 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:45.960637 29638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:45.965225 29648 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [term 1 LEADER]: Becoming Leader. State: Replica: 2bfd285021d040cda9a0fb349f196ed6, State: Running, Role: LEADER
I20260929 02:23:45.965903 29648 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } }
I20260929 02:23:45.968609 29649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bfd285021d040cda9a0fb349f196ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } } }
I20260929 02:23:45.968976 29649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:45.969460 29650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bfd285021d040cda9a0fb349f196ed6. Latest consensus state: current_term: 1 leader_uuid: "2bfd285021d040cda9a0fb349f196ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bfd285021d040cda9a0fb349f196ed6" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 34201 } } }
I20260929 02:23:45.969723 29650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:45.973766 29641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:45.956293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55415 (local address 127.7.170.62:34201)
0929 02:23:45.956482 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:45.956489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:45.956837 (+ 348us) server_negotiation.cc:407] Connection header received
0929 02:23:45.957027 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:45.957030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:45.957088 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:45.957161 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:45.958868 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:45.960046 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:45.961998 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:45.962698 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:45.962826 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:45.966035 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:45.966041 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:45.966049 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:45.966086 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:45.969704 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:45.970716 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:45.970722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:45.970726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:45.970886 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:45.971432 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:45.971437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:45.971441 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:45.971908 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:45.972251 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:45.973203 (+ 952us) server_negotiation.cc:299] Negotiation successful
0929 02:23:45.973395 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260929 02:23:45.979398 29656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:45.983534 29656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:45.991451 29656 catalog_manager.cc:1347] Generated new cluster ID: 25f7558784464f30aab886e91ed50a39
I20260929 02:23:45.991565 29656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:46.007200 29656 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:46.009006 29656 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:46.022292 29656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bfd285021d040cda9a0fb349f196ed6: Generated new TSK 0
I20260929 02:23:46.023120 29656 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:46.035684 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29571
I20260929 02:23:46.037254 29633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:46.127998 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29571
W20260929 02:23:46.143422 29556 connection.cc:535] client connection to 127.7.170.62:34201 recv error: Network error: failed to read from TLS socket (remote: 127.7.170.62:34201): Connection reset by peer (error 104)
2026-09-29T02:23:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1928 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:46 dist-test-slave-8cs2 krb5kdc[29670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:46 dist-test-slave-8cs2 krb5kdc[29670](info): set up 2 sockets
Sep 29 02:23:46 dist-test-slave-8cs2 krb5kdc[29670](info): commencing operation
krb5kdc: starting...
W20260929 02:23:47.165426 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:47 dist-test-slave-8cs2 krb5kdc[29670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:47.440402 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42371
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:42371
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:47.769827 29683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:47.770121 29683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:47.770244 29683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:47.777436 29683 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:47.777534 29683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:47.777601 29683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:47.777684 29683 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:47.777741 29683 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:47.787432 29683 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:42371
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:42371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:47.788708 29683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:47.790019 29683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:47.801471 29691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:47.801471 29689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:47.801523 29688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:47.802426 29683 server_base.cc:1034] running on GCE node
I20260929 02:23:47.803226 29683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:47.804627 29683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:47.805882 29683 hybrid_clock.cc:648] HybridClock initialized: now 1790648627805831 us; error 39 us; skew 500 ppm
I20260929 02:23:47.806198 29683 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:47 dist-test-slave-8cs2 krb5kdc[29670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648627, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:47.809167 29683 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:47.810592 29683 webserver.cc:460] Webserver started at http://127.7.170.62:45295/ using document root <none> and password file <none>
I20260929 02:23:47.811231 29683 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:47.811352 29683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:47.811633 29683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:47.814563 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "e3574b063833450b873993526e45f891"
format_stamp: "Formatted at 2026-09-29 02:23:47 on dist-test-slave-8cs2"
server_key: "0f9412902d6a8e796f754307bf5a9999"
server_key_iv: "c7d8a35b72d4fece90aab435f963ac1a"
server_key_version: "encryptionkey@0"
I20260929 02:23:47.815286 29683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "e3574b063833450b873993526e45f891"
format_stamp: "Formatted at 2026-09-29 02:23:47 on dist-test-slave-8cs2"
server_key: "0f9412902d6a8e796f754307bf5a9999"
server_key_iv: "c7d8a35b72d4fece90aab435f963ac1a"
server_key_version: "encryptionkey@0"
I20260929 02:23:47.820254 29683 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:23:47.823715 29698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:47.825037 29683 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:47.825229 29683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "e3574b063833450b873993526e45f891"
format_stamp: "Formatted at 2026-09-29 02:23:47 on dist-test-slave-8cs2"
server_key: "0f9412902d6a8e796f754307bf5a9999"
server_key_iv: "c7d8a35b72d4fece90aab435f963ac1a"
server_key_version: "encryptionkey@0"
I20260929 02:23:47.825418 29683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:47.844506 29683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:47.848616 29683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:47.848872 29683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:47.865765 29683 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:42371
I20260929 02:23:47.865801 29749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:42371 every 8 connection(s)
I20260929 02:23:47.867089 29683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:47.870644 29750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:47.875888 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29683
I20260929 02:23:47.876078 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:47.876494 7848 external_mini_cluster.cc:1428] Setting key 25be38ba0740a453455f692d9570b3b3
I20260929 02:23:47.880879 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891: Bootstrap starting.
I20260929 02:23:47.884138 29750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:47.885355 29750 log.cc:824] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:47.888540 29750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891: No bootstrap required, opened a new log
I20260929 02:23:47.890014 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:47.878358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60133 (local address 127.7.170.62:42371)
0929 02:23:47.879164 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:47.879175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:47.879214 (+ 39us) server_negotiation.cc:407] Connection header received
0929 02:23:47.880826 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:47.880849 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:47.881279 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:47.881585 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:47.882188 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:47.883682 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:47.885489 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:47.886444 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:47.886622 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:47.887776 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:47.887784 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:47.887797 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:47.888328 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:47.888404 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:47.888429 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:47.888548 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:47.888844 (+ 296us) server_negotiation.cc:299] Negotiation successful
0929 02:23:47.889051 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":385,"threads_started":1}
I20260929 02:23:47.893998 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:47.898190 29750 raft_consensus.cc:348] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } }
I20260929 02:23:47.898630 29750 raft_consensus.cc:374] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:47.898749 29750 raft_consensus.cc:729] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3574b063833450b873993526e45f891, State: Initialized, Role: FOLLOWER
I20260929 02:23:47.899420 29750 consensus_queue.cc:260] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } }
I20260929 02:23:47.899698 29750 raft_consensus.cc:388] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:47.899863 29750 raft_consensus.cc:482] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:47.900027 29750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:47.903010 29750 raft_consensus.cc:504] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } }
I20260929 02:23:47.903549 29750 leader_election.cc:302] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e3574b063833450b873993526e45f891; no voters:
I20260929 02:23:47.904140 29750 leader_election.cc:288] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:47.904245 29760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:47.905937 29753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:47.899015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32799 (local address 127.7.170.62:42371)
0929 02:23:47.899235 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:47.899241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:47.899658 (+ 417us) server_negotiation.cc:407] Connection header received
0929 02:23:47.899830 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:47.899835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:47.899912 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:47.900017 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:47.900592 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:47.901421 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:47.903194 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:47.903675 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:47.903790 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:47.905063 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:47.905070 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:47.905075 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:47.905093 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:47.905157 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:47.905177 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:47.905252 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:47.905529 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 02:23:47.905583 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:47.906008 29760 raft_consensus.cc:686] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [term 1 LEADER]: Becoming Leader. State: Replica: e3574b063833450b873993526e45f891, State: Running, Role: LEADER
I20260929 02:23:47.906520 29750 sys_catalog.cc:564] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:47.906714 29760 consensus_queue.cc:237] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } }
I20260929 02:23:47.909564 29762 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e3574b063833450b873993526e45f891. Latest consensus state: current_term: 1 leader_uuid: "e3574b063833450b873993526e45f891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } } }
I20260929 02:23:47.909948 29762 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:47.910423 29761 sys_catalog.cc:455] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e3574b063833450b873993526e45f891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e3574b063833450b873993526e45f891" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 42371 } } }
I20260929 02:23:47.910625 29761 sys_catalog.cc:458] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:47.913830 29768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:47.918396 29768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:47.926738 29768 catalog_manager.cc:1347] Generated new cluster ID: 7407e78bc7514d65ab5cfd239cb17127
I20260929 02:23:47.926853 29768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:47.943084 29768 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:47.944391 29768 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:47.961338 29768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e3574b063833450b873993526e45f891: Generated new TSK 0
I20260929 02:23:47.962348 29768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:47.977348 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29683
W20260929 02:23:47.978485 29745 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60133
W20260929 02:23:47.979243 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29683: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:47.979390 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29683
W20260929 02:23:47.993680 29668 connection.cc:535] client connection to 127.7.170.62:42371 recv error: Network error: failed to read from TLS socket (remote: 127.7.170.62:42371): Connection reset by peer (error 104)
2026-09-29T02:23:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1850 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:48 dist-test-slave-8cs2 krb5kdc[29781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:48 dist-test-slave-8cs2 krb5kdc[29781](info): set up 2 sockets
Sep 29 02:23:48 dist-test-slave-8cs2 krb5kdc[29781](info): commencing operation
krb5kdc: starting...
W20260929 02:23:49.001544 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.989s user 0.005s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:49 dist-test-slave-8cs2 krb5kdc[29781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:49.288142 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:37753
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:38991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:37753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:49.661015 29794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:49.661374 29794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:49.661520 29794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:49.669329 29794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:49.669461 29794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:49.669520 29794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:49.669617 29794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:49.669673 29794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:49.680368 29794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:38991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:37753
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:37753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:49.682036 29794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:49.683657 29794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:49.696853 29802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:49.696910 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:49.697199 29799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:49.697916 29794 server_base.cc:1034] running on GCE node
I20260929 02:23:49.698784 29794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:49.700439 29794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:49.701666 29794 hybrid_clock.cc:648] HybridClock initialized: now 1790648629701622 us; error 39 us; skew 500 ppm
I20260929 02:23:49.702042 29794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:49 dist-test-slave-8cs2 krb5kdc[29781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648629, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:49.714020 29794 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:49.715780 29794 webserver.cc:460] Webserver started at http://127.7.170.62:39873/ using document root <none> and password file <none>
I20260929 02:23:49.716573 29794 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:49.716696 29794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:49.717075 29794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:49.720508 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0"
format_stamp: "Formatted at 2026-09-29 02:23:49 on dist-test-slave-8cs2"
server_key: "278b27888ff0fd8d2e3ea683ec1abf1d"
server_key_iv: "dabb5ded65e89b387e73edfedfab7f61"
server_key_version: "encryptionkey@0"
I20260929 02:23:49.721369 29794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0"
format_stamp: "Formatted at 2026-09-29 02:23:49 on dist-test-slave-8cs2"
server_key: "278b27888ff0fd8d2e3ea683ec1abf1d"
server_key_iv: "dabb5ded65e89b387e73edfedfab7f61"
server_key_version: "encryptionkey@0"
I20260929 02:23:49.727006 29794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 02:23:49.730979 29809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:49.732498 29794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260929 02:23:49.732661 29794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0"
format_stamp: "Formatted at 2026-09-29 02:23:49 on dist-test-slave-8cs2"
server_key: "278b27888ff0fd8d2e3ea683ec1abf1d"
server_key_iv: "dabb5ded65e89b387e73edfedfab7f61"
server_key_version: "encryptionkey@0"
I20260929 02:23:49.732847 29794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:49.739115 29794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:49.743263 29794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:49.743516 29794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:49.762494 29794 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:37753
I20260929 02:23:49.762526 29860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:37753 every 8 connection(s)
I20260929 02:23:49.763995 29794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:49.768253 29861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:49.769323 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29794
I20260929 02:23:49.769498 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:49.769985 7848 external_mini_cluster.cc:1428] Setting key 0da10da2a5dad7a704148ca9c6309537
I20260929 02:23:49.778103 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:49.771898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34181 (local address 127.7.170.62:37753)
0929 02:23:49.772703 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:49.772716 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:49.772771 (+ 55us) server_negotiation.cc:407] Connection header received
0929 02:23:49.774593 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:49.774625 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:49.775000 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:49.775357 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:49.776485 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:49.776492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:49.776506 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:49.776520 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:49.776581 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:49.776615 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:49.776781 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:49.776883 (+ 102us) server_negotiation.cc:299] Negotiation successful
0929 02:23:49.777116 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":359,"threads_started":1}
I20260929 02:23:49.781960 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:49.782269 29861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0: Bootstrap starting.
I20260929 02:23:49.785641 29861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:49.786883 29861 log.cc:824] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:49.789094 29864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:49.786560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42791 (local address 127.7.170.62:37753)
0929 02:23:49.786747 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:49.786753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:49.787162 (+ 409us) server_negotiation.cc:407] Connection header received
0929 02:23:49.787297 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:49.787303 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:49.787381 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:49.787493 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:49.788487 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:49.788493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:49.788497 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:49.788514 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:49.788558 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:49.788576 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:49.788689 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:49.788735 (+ 46us) server_negotiation.cc:299] Negotiation successful
0929 02:23:49.788789 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:23:49.790252 29861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0: No bootstrap required, opened a new log
I20260929 02:23:49.798249 29861 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } }
I20260929 02:23:49.798746 29861 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:49.798848 29861 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0f7a98f4da4f8cb40b38980dfbbfa0, State: Initialized, Role: FOLLOWER
I20260929 02:23:49.799402 29861 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } }
I20260929 02:23:49.799621 29861 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:49.799723 29861 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:49.799844 29861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:49.802057 29861 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } }
I20260929 02:23:49.802525 29861 leader_election.cc:302] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe0f7a98f4da4f8cb40b38980dfbbfa0; no voters:
I20260929 02:23:49.803062 29861 leader_election.cc:288] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:49.803185 29871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:49.804112 29871 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [term 1 LEADER]: Becoming Leader. State: Replica: fe0f7a98f4da4f8cb40b38980dfbbfa0, State: Running, Role: LEADER
I20260929 02:23:49.804626 29871 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } }
I20260929 02:23:49.805083 29861 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:49.807790 29873 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe0f7a98f4da4f8cb40b38980dfbbfa0. Latest consensus state: current_term: 1 leader_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } } }
I20260929 02:23:49.807855 29872 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0f7a98f4da4f8cb40b38980dfbbfa0" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 37753 } } }
I20260929 02:23:49.808379 29873 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:49.808399 29872 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:49.810902 29880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:49.816493 29880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:49.826156 29880 catalog_manager.cc:1347] Generated new cluster ID: e38e7dfbf8864c74b5db12cd939a1cd9
I20260929 02:23:49.826269 29880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:49.843863 29880 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:49.844319 29880 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:49.851176 29880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe0f7a98f4da4f8cb40b38980dfbbfa0: Generated new TSK 0
I20260929 02:23:49.852010 29880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:49.867774 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29794
W20260929 02:23:49.869096 29856 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34181
W20260929 02:23:49.869840 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29794: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:49.869969 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29794
2026-09-29T02:23:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1893 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:49 dist-test-slave-8cs2 krb5kdc[29892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:49 dist-test-slave-8cs2 krb5kdc[29892](info): set up 2 sockets
Sep 29 02:23:49 dist-test-slave-8cs2 krb5kdc[29892](info): commencing operation
krb5kdc: starting...
W20260929 02:23:50.909530 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:51.168149 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38893
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:51.509378 29905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:51.509652 29905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:51.509752 29905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:51.516944 29905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:51.517055 29905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:51.517098 29905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:51.517158 29905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:51.517205 29905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:51.527247 29905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:51.528458 29905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:51.529742 29905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:51.540906 29910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:51.540969 29913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:51.540997 29911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:51.542302 29905 server_base.cc:1034] running on GCE node
I20260929 02:23:51.543023 29905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:51.544409 29905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:51.545629 29905 hybrid_clock.cc:648] HybridClock initialized: now 1790648631545587 us; error 30 us; skew 500 ppm
I20260929 02:23:51.546026 29905 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[29892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648631, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:51.549345 29905 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:51.550894 29905 webserver.cc:460] Webserver started at http://127.7.170.62:35271/ using document root <none> and password file <none>
I20260929 02:23:51.551577 29905 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:51.551704 29905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:51.551996 29905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:51.554994 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "807b2124c477437a8d35b025721715eb"
format_stamp: "Formatted at 2026-09-29 02:23:51 on dist-test-slave-8cs2"
server_key: "8304f6e5fd08dfc425a612cbf3fcd274"
server_key_iv: "d19f9b2d4e981aae63e6f773d918fc6d"
server_key_version: "encryptionkey@0"
I20260929 02:23:51.555747 29905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "807b2124c477437a8d35b025721715eb"
format_stamp: "Formatted at 2026-09-29 02:23:51 on dist-test-slave-8cs2"
server_key: "8304f6e5fd08dfc425a612cbf3fcd274"
server_key_iv: "d19f9b2d4e981aae63e6f773d918fc6d"
server_key_version: "encryptionkey@0"
I20260929 02:23:51.560737 29905 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:23:51.564150 29920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:51.565505 29905 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:51.565660 29905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "807b2124c477437a8d35b025721715eb"
format_stamp: "Formatted at 2026-09-29 02:23:51 on dist-test-slave-8cs2"
server_key: "8304f6e5fd08dfc425a612cbf3fcd274"
server_key_iv: "d19f9b2d4e981aae63e6f773d918fc6d"
server_key_version: "encryptionkey@0"
I20260929 02:23:51.565858 29905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:51.579285 29905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:51.583060 29905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:51.583305 29905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:51.599933 29905 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38893
I20260929 02:23:51.599964 29971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38893 every 8 connection(s)
I20260929 02:23:51.601207 29905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:51.601680 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 29905
I20260929 02:23:51.601840 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:51.602396 7848 external_mini_cluster.cc:1428] Setting key a92edccfd722f5ee0f8c38e1d9d6f85e
I20260929 02:23:51.605492 29972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[29892](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:51.616832 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb: Bootstrap starting.
I20260929 02:23:51.620066 29972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:51.621240 29972 log.cc:824] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:51.623800 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:51.604229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55147 (local address 127.7.170.62:38893)
0929 02:23:51.605032 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:51.605044 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:51.605086 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:23:51.606762 (+ 1676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:51.606785 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:51.607163 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:51.607516 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:51.609001 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:51.610332 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:51.611586 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:51.612266 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:51.612444 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:51.617089 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:51.617097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:51.617110 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:51.617148 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:51.619969 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:51.620732 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:51.620738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:51.620743 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:51.620904 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:51.621282 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:51.621285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:51.621288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:51.621882 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:51.622161 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:51.622532 (+ 371us) server_negotiation.cc:299] Negotiation successful
0929 02:23:51.622869 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":340,"threads_started":1}
I20260929 02:23:51.624075 29972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb: No bootstrap required, opened a new log
I20260929 02:23:51.627513 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:51.633949 29972 raft_consensus.cc:348] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } }
I20260929 02:23:51.634347 29972 raft_consensus.cc:374] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:51.634452 29972 raft_consensus.cc:729] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 807b2124c477437a8d35b025721715eb, State: Initialized, Role: FOLLOWER
I20260929 02:23:51.635084 29972 consensus_queue.cc:260] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } }
I20260929 02:23:51.635336 29972 raft_consensus.cc:388] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:51.635464 29972 raft_consensus.cc:482] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:51.635624 29972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:51.638521 29972 raft_consensus.cc:504] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } }
I20260929 02:23:51.639097 29972 leader_election.cc:302] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 807b2124c477437a8d35b025721715eb; no voters:
I20260929 02:23:51.639767 29972 leader_election.cc:288] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:51.639916 29982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:51.644526 29972 sys_catalog.cc:564] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:51.644971 29982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [term 1 LEADER]: Becoming Leader. State: Replica: 807b2124c477437a8d35b025721715eb, State: Running, Role: LEADER
I20260929 02:23:51.645645 29982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } }
I20260929 02:23:51.648756 29984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 807b2124c477437a8d35b025721715eb. Latest consensus state: current_term: 1 leader_uuid: "807b2124c477437a8d35b025721715eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } } }
I20260929 02:23:51.649180 29984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:51.653353 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:51.633686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49819 (local address 127.7.170.62:38893)
0929 02:23:51.633854 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:51.633859 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:51.633876 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:23:51.633921 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:51.633926 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:51.634000 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:51.634086 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:51.635826 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:51.636977 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:51.638669 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:51.639282 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:51.639397 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:51.642978 (+ 3581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:51.642984 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:51.642989 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:51.643042 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:51.647346 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:51.648345 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:51.648351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:51.648355 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:51.648596 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:51.650707 (+ 2111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:51.650713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:51.650717 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:51.651140 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:51.652508 (+ 1368us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:51.652660 (+ 152us) server_negotiation.cc:299] Negotiation successful
0929 02:23:51.652901 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260929 02:23:51.654973 29983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "807b2124c477437a8d35b025721715eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "807b2124c477437a8d35b025721715eb" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38893 } } }
I20260929 02:23:51.655287 29983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:51.656446 29991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:51.661724 29991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:51.670431 29991 catalog_manager.cc:1347] Generated new cluster ID: 04cd091d24a6477a9ec69e7579305ecc
I20260929 02:23:51.670539 29991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:51.687407 29991 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:51.689221 29991 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:51.706677 29991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 807b2124c477437a8d35b025721715eb: Generated new TSK 0
I20260929 02:23:51.707748 29991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:51.716066 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 29905
I20260929 02:23:51.717761 29967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:51.806704 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 29905
2026-09-29T02:23:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1933 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[30004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[30004](info): set up 2 sockets
Sep 29 02:23:51 dist-test-slave-8cs2 krb5kdc[30004](info): commencing operation
krb5kdc: starting...
W20260929 02:23:52.817417 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.979s user 0.004s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:52 dist-test-slave-8cs2 krb5kdc[30004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:53.074472 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44179
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:45677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:44179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:53.390758 30017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:53.391027 30017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:53.391163 30017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:53.398190 30017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:53.398291 30017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:53.398363 30017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:53.398612 30017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:53.398691 30017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:53.408766 30017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:45677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:44179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:44179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:53.410058 30017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:53.411339 30017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:53.422223 30022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:53.422261 30023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:53.422287 30025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:53.423252 30017 server_base.cc:1034] running on GCE node
I20260929 02:23:53.423962 30017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:53.425318 30017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:53.426520 30017 hybrid_clock.cc:648] HybridClock initialized: now 1790648633426466 us; error 42 us; skew 500 ppm
I20260929 02:23:53.426849 30017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:53 dist-test-slave-8cs2 krb5kdc[30004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648633, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:53.429958 30017 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:53.431308 30017 webserver.cc:460] Webserver started at http://127.7.170.62:38277/ using document root <none> and password file <none>
I20260929 02:23:53.431947 30017 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:53.432060 30017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:53.432337 30017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:53.435251 30017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "f498db217b4c4505b232d2c68dbd55be"
format_stamp: "Formatted at 2026-09-29 02:23:53 on dist-test-slave-8cs2"
server_key: "4ea5a36ee080217db618c2d5e8fe9d0a"
server_key_iv: "61776f20c9f442a2231beda239c861f3"
server_key_version: "encryptionkey@0"
I20260929 02:23:53.435986 30017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "f498db217b4c4505b232d2c68dbd55be"
format_stamp: "Formatted at 2026-09-29 02:23:53 on dist-test-slave-8cs2"
server_key: "4ea5a36ee080217db618c2d5e8fe9d0a"
server_key_iv: "61776f20c9f442a2231beda239c861f3"
server_key_version: "encryptionkey@0"
I20260929 02:23:53.441015 30017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260929 02:23:53.444325 30032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:53.445652 30017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:53.445820 30017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "f498db217b4c4505b232d2c68dbd55be"
format_stamp: "Formatted at 2026-09-29 02:23:53 on dist-test-slave-8cs2"
server_key: "4ea5a36ee080217db618c2d5e8fe9d0a"
server_key_iv: "61776f20c9f442a2231beda239c861f3"
server_key_version: "encryptionkey@0"
I20260929 02:23:53.445998 30017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:53.464771 30017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:53.469213 30017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:53.469483 30017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:53.487026 30017 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:44179
I20260929 02:23:53.487046 30083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:44179 every 8 connection(s)
I20260929 02:23:53.488447 30017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:53.489301 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30017
I20260929 02:23:53.489487 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:53.490157 7848 external_mini_cluster.cc:1428] Setting key 648f8944caaa0b579c32e8ffc2d4b720
I20260929 02:23:53.493376 30084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:53.503208 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:53.491997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34045 (local address 127.7.170.62:44179)
0929 02:23:53.492761 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:53.492773 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:53.492813 (+ 40us) server_negotiation.cc:407] Connection header received
0929 02:23:53.494564 (+ 1751us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:53.494588 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:53.495014 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:53.495450 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:53.496017 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:53.497585 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:53.498874 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:53.499519 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:53.499692 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:53.500849 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:53.500857 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:53.500874 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:53.501497 (+ 623us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:53.501573 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:53.501597 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:53.501706 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:53.501945 (+ 239us) server_negotiation.cc:299] Negotiation successful
0929 02:23:53.502199 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":352,"threads_started":1}
I20260929 02:23:53.504355 30084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be: Bootstrap starting.
I20260929 02:23:53.506807 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:53.507638 30084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:53.508585 30084 log.cc:824] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:53.511148 30084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be: No bootstrap required, opened a new log
I20260929 02:23:53.519251 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:53.512952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34067 (local address 127.7.170.62:44179)
0929 02:23:53.513117 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:53.513124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:53.513581 (+ 457us) server_negotiation.cc:407] Connection header received
0929 02:23:53.513763 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:53.513767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:53.513826 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:53.513912 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:53.514496 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:53.515205 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:53.517012 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:53.517477 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:53.517571 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:53.518605 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:53.518609 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:53.518612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:53.518630 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:53.518670 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:53.518686 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:53.518752 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:53.518889 (+ 137us) server_negotiation.cc:299] Negotiation successful
0929 02:23:53.518954 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260929 02:23:53.520218 30084 raft_consensus.cc:348] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } }
I20260929 02:23:53.520569 30084 raft_consensus.cc:374] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:53.520655 30084 raft_consensus.cc:729] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f498db217b4c4505b232d2c68dbd55be, State: Initialized, Role: FOLLOWER
I20260929 02:23:53.521198 30084 consensus_queue.cc:260] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } }
I20260929 02:23:53.521415 30084 raft_consensus.cc:388] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:53.521533 30084 raft_consensus.cc:482] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:53.521689 30084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:53.523803 30084 raft_consensus.cc:504] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } }
I20260929 02:23:53.524276 30084 leader_election.cc:302] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f498db217b4c4505b232d2c68dbd55be; no voters:
I20260929 02:23:53.524816 30084 leader_election.cc:288] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:53.524947 30094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:53.525521 30094 raft_consensus.cc:686] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [term 1 LEADER]: Becoming Leader. State: Replica: f498db217b4c4505b232d2c68dbd55be, State: Running, Role: LEADER
I20260929 02:23:53.526103 30094 consensus_queue.cc:237] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } }
I20260929 02:23:53.526878 30084 sys_catalog.cc:564] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:53.528627 30096 sys_catalog.cc:455] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [sys.catalog]: SysCatalogTable state changed. Reason: New leader f498db217b4c4505b232d2c68dbd55be. Latest consensus state: current_term: 1 leader_uuid: "f498db217b4c4505b232d2c68dbd55be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } } }
I20260929 02:23:53.529000 30096 sys_catalog.cc:458] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:53.529475 30095 sys_catalog.cc:455] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f498db217b4c4505b232d2c68dbd55be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f498db217b4c4505b232d2c68dbd55be" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 44179 } } }
I20260929 02:23:53.529753 30095 sys_catalog.cc:458] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:53.531865 30101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:53.536724 30101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:53.545258 30101 catalog_manager.cc:1347] Generated new cluster ID: 789c4888914b40ea9be41e46c20d3118
I20260929 02:23:53.545372 30101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:53.587949 30101 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:53.589326 30101 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:53.603003 30101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f498db217b4c4505b232d2c68dbd55be: Generated new TSK 0
I20260929 02:23:53.603909 30101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:53.623474 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30017
W20260929 02:23:53.624850 30079 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34045
W20260929 02:23:53.625584 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30017: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:53.625720 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30017
2026-09-29T02:23:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1819 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:53 dist-test-slave-8cs2 krb5kdc[30115](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:53 dist-test-slave-8cs2 krb5kdc[30115](info): set up 2 sockets
Sep 29 02:23:53 dist-test-slave-8cs2 krb5kdc[30115](info): commencing operation
krb5kdc: starting...
W20260929 02:23:54.649547 7848 mini_kdc.cc:121] Time spent starting KDC: real 0.991s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:54 dist-test-slave-8cs2 krb5kdc[30115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:54.920477 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:45207
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:45207
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:55.247242 30128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:55.247547 30128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:55.247670 30128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:55.254990 30128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:55.255100 30128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:55.255170 30128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:55.255256 30128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:55.255312 30128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:55.265365 30128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:45207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:45207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:55.266752 30128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:55.268077 30128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:55.279430 30136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:55.279480 30133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:55.279502 30134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:55.280405 30128 server_base.cc:1034] running on GCE node
I20260929 02:23:55.281234 30128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:55.282647 30128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:55.283900 30128 hybrid_clock.cc:648] HybridClock initialized: now 1790648635283807 us; error 60 us; skew 500 ppm
I20260929 02:23:55.284253 30128 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:55 dist-test-slave-8cs2 krb5kdc[30115](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648635, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:55.291988 30128 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:55.293598 30128 webserver.cc:460] Webserver started at http://127.7.170.62:45157/ using document root <none> and password file <none>
I20260929 02:23:55.294293 30128 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:55.294417 30128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:55.294711 30128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:55.297873 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "f3e1c9954ba84a22bde76716d5197a9c"
format_stamp: "Formatted at 2026-09-29 02:23:55 on dist-test-slave-8cs2"
server_key: "5ffae17fe3afa97712f2f59aaa195d06"
server_key_iv: "9f13f99afebf964558c18126a070f14f"
server_key_version: "encryptionkey@0"
I20260929 02:23:55.298656 30128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "f3e1c9954ba84a22bde76716d5197a9c"
format_stamp: "Formatted at 2026-09-29 02:23:55 on dist-test-slave-8cs2"
server_key: "5ffae17fe3afa97712f2f59aaa195d06"
server_key_iv: "9f13f99afebf964558c18126a070f14f"
server_key_version: "encryptionkey@0"
I20260929 02:23:55.304102 30128 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260929 02:23:55.307765 30143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:55.309191 30128 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260929 02:23:55.309360 30128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "f3e1c9954ba84a22bde76716d5197a9c"
format_stamp: "Formatted at 2026-09-29 02:23:55 on dist-test-slave-8cs2"
server_key: "5ffae17fe3afa97712f2f59aaa195d06"
server_key_iv: "9f13f99afebf964558c18126a070f14f"
server_key_version: "encryptionkey@0"
I20260929 02:23:55.309561 30128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:55.315297 30128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:55.319617 30128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:55.319888 30128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:55.337728 30128 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:45207
I20260929 02:23:55.337762 30194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:45207 every 8 connection(s)
I20260929 02:23:55.339197 30128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:55.343037 30195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:55.346448 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30128
I20260929 02:23:55.346626 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:55.347065 7848 external_mini_cluster.cc:1428] Setting key 75d0cb55c985835d38d8dfb08033772c
I20260929 02:23:55.354853 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:55.348848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58197 (local address 127.7.170.62:45207)
0929 02:23:55.349670 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:55.349683 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:55.349727 (+ 44us) server_negotiation.cc:407] Connection header received
0929 02:23:55.351407 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:55.351434 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:55.351832 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:55.352183 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:55.353289 (+ 1106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:55.353295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:55.353309 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:55.353323 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:55.353394 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:55.353415 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:55.353585 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:55.353682 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 02:23:55.353862 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":351,"threads_started":1}
I20260929 02:23:55.355401 30195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c: Bootstrap starting.
I20260929 02:23:55.358553 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:55.358695 30195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:55.359709 30195 log.cc:824] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:55.363101 30195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c: No bootstrap required, opened a new log
I20260929 02:23:55.365317 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:55.362739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48023 (local address 127.7.170.62:45207)
0929 02:23:55.362950 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:55.362957 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:55.363411 (+ 454us) server_negotiation.cc:407] Connection header received
0929 02:23:55.363591 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:55.363594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:55.363647 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:55.363728 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:55.364650 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:55.364654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:55.364657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0929 02:23:55.364670 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:55.364708 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0929 02:23:55.364723 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:55.364808 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:55.364990 (+ 182us) server_negotiation.cc:299] Negotiation successful
0929 02:23:55.365040 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260929 02:23:55.371309 30195 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } }
I20260929 02:23:55.371709 30195 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260929 02:23:55.371799 30195 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3e1c9954ba84a22bde76716d5197a9c, State: Initialized, Role: FOLLOWER
I20260929 02:23:55.372354 30195 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } }
I20260929 02:23:55.372596 30195 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260929 02:23:55.372721 30195 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260929 02:23:55.372874 30195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:55.375046 30195 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } }
I20260929 02:23:55.375530 30195 leader_election.cc:302] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3e1c9954ba84a22bde76716d5197a9c; no voters:
I20260929 02:23:55.376076 30195 leader_election.cc:288] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260929 02:23:55.376227 30205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:55.376842 30205 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [term 1 LEADER]: Becoming Leader. State: Replica: f3e1c9954ba84a22bde76716d5197a9c, State: Running, Role: LEADER
I20260929 02:23:55.377525 30205 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } }
I20260929 02:23:55.378212 30195 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:55.380683 30206 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3e1c9954ba84a22bde76716d5197a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } } }
I20260929 02:23:55.381134 30206 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:55.381091 30207 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3e1c9954ba84a22bde76716d5197a9c. Latest consensus state: current_term: 1 leader_uuid: "f3e1c9954ba84a22bde76716d5197a9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3e1c9954ba84a22bde76716d5197a9c" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 45207 } } }
I20260929 02:23:55.381392 30207 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:55.382920 30213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:55.387313 30213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:55.396478 30213 catalog_manager.cc:1347] Generated new cluster ID: 5693db28f98a4715853bb2391352b56b
I20260929 02:23:55.396580 30213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:55.407783 30213 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:55.408179 30213 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:55.417969 30213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3e1c9954ba84a22bde76716d5197a9c: Generated new TSK 0
I20260929 02:23:55.418828 30213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260929 02:23:55.432387 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30128
W20260929 02:23:55.433794 30190 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58197
W20260929 02:23:55.434535 7848 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30128: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260929 02:23:55.434665 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30128
2026-09-29T02:23:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1808 ms)
[----------] 18 tests from AuthTokenIssuingTest (33315 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:23:55 dist-test-slave-8cs2 krb5kdc[30226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:23:55 dist-test-slave-8cs2 krb5kdc[30226](info): set up 2 sockets
Sep 29 02:23:55 dist-test-slave-8cs2 krb5kdc[30226](info): commencing operation
krb5kdc: starting...
W20260929 02:23:56.477667 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:23:56 dist-test-slave-8cs2 krb5kdc[30226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:23:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:23:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:23:56.746260 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36349
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:23:57.067750 30239 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:23:57.068096 30239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:57.068192 30239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:57.068305 30239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:57.075493 30239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:57.075594 30239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:57.075676 30239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:57.075738 30239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:57.075829 30239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:57.085721 30239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:36349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:57.087074 30239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:57.088392 30239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:23:57.099352 30247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:57.099397 30244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:57.099429 30245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:57.100225 30239 server_base.cc:1034] running on GCE node
I20260929 02:23:57.101029 30239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:57.102392 30239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:57.103619 30239 hybrid_clock.cc:648] HybridClock initialized: now 1790648637103559 us; error 27 us; skew 500 ppm
I20260929 02:23:57.103958 30239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:57.107033 30239 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:23:57.108387 30239 webserver.cc:460] Webserver started at http://127.7.170.62:36029/ using document root <none> and password file <none>
I20260929 02:23:57.109090 30239 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:57.109193 30239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:57.109539 30239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:57.112474 30239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "af447091f25d432d9f54abda8749ba46"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "906fe17944ffa23c7f4993043fe42415"
server_key_iv: "bebe250ef7bbb6a05fca4c9fbbd37a5b"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.113250 30239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "af447091f25d432d9f54abda8749ba46"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "906fe17944ffa23c7f4993043fe42415"
server_key_iv: "bebe250ef7bbb6a05fca4c9fbbd37a5b"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.118258 30239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:23:57.121656 30254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:57.122959 30239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:23:57.123118 30239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "af447091f25d432d9f54abda8749ba46"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "906fe17944ffa23c7f4993043fe42415"
server_key_iv: "bebe250ef7bbb6a05fca4c9fbbd37a5b"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.123329 30239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:57.145304 30239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:57.149212 30239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:57.149453 30239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:57.166098 30239 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:36349
I20260929 02:23:57.166119 30305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:36349 every 8 connection(s)
I20260929 02:23:57.167737 30239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:23:57.171233 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30239
I20260929 02:23:57.171434 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:23:57.172026 30306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:57.172070 7848 external_mini_cluster.cc:1428] Setting key ba45cb536ed588165563b92e15ce0e3f
I20260929 02:23:57.178314 30306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.61@KRBTEST.COM" created.
I20260929 02:23:57.199543 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.181387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36349 (local address 127.0.0.1:42174)
0929 02:23:57.182217 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.182262 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:57.184156 (+ 1894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:57.184971 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:57.184978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:57.185649 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:57.185981 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.185998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.187483 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.187489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.188549 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.188561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.189490 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.189497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.189664 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.190233 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:57.190260 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:57.192276 (+ 2016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:57.195390 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.195401 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.195414 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.195839 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.196198 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.196203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.196207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.196364 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.197237 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:57.197245 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:57.197447 (+ 202us) client_negotiation.cc:769] Sending connection context
0929 02:23:57.197666 (+ 219us) client_negotiation.cc:240] Negotiation successful
0929 02:23:57.198247 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":130,"thread_start_us":350,"threads_started":1}
I20260929 02:23:57.199558 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.181388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42174 (local address 127.7.170.62:36349)
0929 02:23:57.182972 (+ 1584us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:57.182980 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:57.183030 (+ 50us) server_negotiation.cc:407] Connection header received
0929 02:23:57.184514 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:57.184520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:57.184731 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:57.184915 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:57.186174 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.187342 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.188680 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.189275 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.189452 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.192487 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:57.192496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:57.192506 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:57.192537 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:57.195278 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.195952 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.195959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.195961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.196099 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.196504 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.196509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.196511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.197094 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:57.197305 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:57.197813 (+ 508us) server_negotiation.cc:299] Negotiation successful
0929 02:23:57.198094 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1380,"thread_start_us":1168,"threads_started":1}
I20260929 02:23:57.202638 30306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:23:57.203725 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.203337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:43954)
0929 02:23:57.203517 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.203615 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260929 02:23:57.203980 30258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
I20260929 02:23:57.205448 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.205127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:43970)
0929 02:23:57.205257 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.205323 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:23:57.205616 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:33517: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
I20260929 02:23:57.227339 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:33517
--webserver_interface=127.7.170.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:23:57.255079 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } attempt: 1
I20260929 02:23:57.256366 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.255805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:43972)
0929 02:23:57.256048 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.256190 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260929 02:23:57.257428 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.257008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:43982)
0929 02:23:57.257148 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.257232 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:23:57.257612 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:33517: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
I20260929 02:23:57.325865 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } attempt: 2
I20260929 02:23:57.327119 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.326585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:43994)
0929 02:23:57.326826 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.326940 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 02:23:57.328168 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.327758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44010)
0929 02:23:57.327908 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.328006 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:23:57.328377 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:33517: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
I20260929 02:23:57.419643 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } attempt: 3
I20260929 02:23:57.420846 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.420325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44024)
0929 02:23:57.420564 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.420673 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 02:23:57.421844 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.421466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44040)
0929 02:23:57.421605 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.421687 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:23:57.422012 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:33517: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
W20260929 02:23:57.545018 30313 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:23:57.545377 30313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:57.545450 30313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:57.545529 30313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:57.552901 30313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:57.553035 30313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:57.553107 30313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:57.553157 30313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:57.553216 30313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:57.563064 30313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:33517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.170.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:57.564251 30313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:57.565264 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } attempt: 4
I20260929 02:23:57.565590 30313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:23:57.566545 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.565967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44048)
0929 02:23:57.566275 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.566391 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260929 02:23:57.567502 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.567124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44054)
0929 02:23:57.567263 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.567363 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260929 02:23:57.567703 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:33517: Network error: Client connection negotiation failed: client connection to 127.7.170.61:33517: connect: Connection refused (error 111)
W20260929 02:23:57.576817 30319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:57.576871 30318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:57.576886 30321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:57.577647 30313 server_base.cc:1034] running on GCE node
I20260929 02:23:57.578392 30313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:57.579661 30313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:57.580904 30313 hybrid_clock.cc:648] HybridClock initialized: now 1790648637580814 us; error 34 us; skew 500 ppm
I20260929 02:23:57.581226 30313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:57.584146 30313 init.cc:377] Logged in from keytab as kudu/127.7.170.61@KRBTEST.COM (short username kudu)
I20260929 02:23:57.585449 30313 webserver.cc:460] Webserver started at http://127.7.170.61:33143/ using document root <none> and password file <none>
I20260929 02:23:57.586097 30313 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:57.586186 30313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:57.586460 30313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:57.589268 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data/instance:
uuid: "23a8c69c96154ec7ac2b2553b2341f80"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "47db92235164062dc0fdf174a2168835"
server_key_iv: "155731cbe0c277ef34008f1a5be89d1c"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.589986 30313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal/instance:
uuid: "23a8c69c96154ec7ac2b2553b2341f80"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "47db92235164062dc0fdf174a2168835"
server_key_iv: "155731cbe0c277ef34008f1a5be89d1c"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.594622 30313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260929 02:23:57.597754 30328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:57.598992 30313 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260929 02:23:57.599136 30313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "23a8c69c96154ec7ac2b2553b2341f80"
format_stamp: "Formatted at 2026-09-29 02:23:57 on dist-test-slave-8cs2"
server_key: "47db92235164062dc0fdf174a2168835"
server_key_iv: "155731cbe0c277ef34008f1a5be89d1c"
server_key_version: "encryptionkey@0"
I20260929 02:23:57.599325 30313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:57.625065 30313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:57.628988 30313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:57.629221 30313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:57.645924 30313 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.61:33517
I20260929 02:23:57.645984 30379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.61:33517 every 8 connection(s)
I20260929 02:23:57.647182 30313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
I20260929 02:23:57.650730 30380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:57.652181 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30313
I20260929 02:23:57.652340 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-1/wal/instance
I20260929 02:23:57.652734 7848 external_mini_cluster.cc:1428] Setting key 6df1b8097b4e2c07ead7db5e883ca21f
I20260929 02:23:57.657120 30380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.60@KRBTEST.COM; defaulting to no policy
I20260929 02:23:57.677193 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.659642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.7.170.62:36349)
0929 02:23:57.659844 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:57.659851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:57.660960 (+ 1109us) server_negotiation.cc:407] Connection header received
0929 02:23:57.662805 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:57.662812 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:57.662904 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:57.663019 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:57.664445 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.665391 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.667087 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.667648 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.667783 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.671284 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:57.671289 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:57.671292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:57.671326 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:57.673839 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.674765 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.674770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.674772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.674913 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.675373 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.675377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.675379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.675669 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:57.675782 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:57.676703 (+ 921us) server_negotiation.cc:299] Negotiation successful
0929 02:23:57.676920 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260929 02:23:57.678172 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.660023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36349 (local address 127.0.0.1:42188)
0929 02:23:57.660784 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.660836 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:57.662539 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:57.663202 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:57.663209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:57.663709 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:57.664262 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.664278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.665531 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.665534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.666928 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.666941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.667836 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.667841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.668070 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.668686 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:57.668709 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:57.671055 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:57.673988 (+ 2933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.673999 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.674018 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.674621 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.675025 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.675030 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.675034 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.675264 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.675796 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:57.675805 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:57.676291 (+ 486us) client_negotiation.cc:769] Sending connection context
0929 02:23:57.676615 (+ 324us) client_negotiation.cc:240] Negotiation successful
0929 02:23:57.677084 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":326,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.60@KRBTEST.COM" created.
I20260929 02:23:57.680009 30380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:23:57.694383 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.680718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44064)
0929 02:23:57.680919 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.680959 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:57.681065 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:57.682135 (+ 1070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:57.682139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:57.682481 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:57.682624 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.682632 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.684307 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.684310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.685224 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.685236 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.686237 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.686241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.686360 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.686898 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:57.686915 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:57.688833 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:57.691829 (+ 2996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.691833 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.691837 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.692253 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.692613 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.692617 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.692620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.692780 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.693387 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:57.693391 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:57.693540 (+ 149us) client_negotiation.cc:769] Sending connection context
0929 02:23:57.693700 (+ 160us) client_negotiation.cc:240] Negotiation successful
0929 02:23:57.693983 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260929 02:23:57.694429 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.680973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44064 (local address 127.7.170.61:33517)
0929 02:23:57.681626 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:57.681634 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:57.681663 (+ 29us) server_negotiation.cc:407] Connection header received
0929 02:23:57.681713 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:57.681720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:57.681953 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:57.682096 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:57.682781 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.684170 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.685353 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.686102 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.686215 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.689058 (+ 2843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:57.689073 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:57.689092 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:57.689126 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:57.691700 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.692388 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.692395 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.692397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.692509 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.692877 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.692880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.692882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.693256 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:57.693436 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:57.693810 (+ 374us) server_negotiation.cc:299] Negotiation successful
0929 02:23:57.694090 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":495,"thread_start_us":343,"threads_started":1}
I20260929 02:23:57.696841 30380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:23:57.698093 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.697550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52714)
0929 02:23:57.697741 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.697915 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260929 02:23:57.698367 30329 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:57.699910 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.699530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52716)
0929 02:23:57.699686 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.699781 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 02:23:57.700140 30380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
I20260929 02:23:57.711427 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:37385
--webserver_interface=127.7.170.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:23:57.749440 30380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 1
I20260929 02:23:57.750830 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.750297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52720)
0929 02:23:57.750518 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.750646 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260929 02:23:57.751837 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.751460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52724)
0929 02:23:57.751603 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.751709 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260929 02:23:57.752054 30380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:57.820302 30380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 2
I20260929 02:23:57.821558 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.821056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52726)
0929 02:23:57.821264 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.821371 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260929 02:23:57.822539 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.822165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52742)
0929 02:23:57.822323 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.822409 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 02:23:57.822739 30380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:57.867010 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } attempt: 5
Sep 29 02:23:57 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:23:57.880689 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.867872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44076 (local address 127.7.170.61:33517)
0929 02:23:57.868048 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:57.868052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:57.868121 (+ 69us) server_negotiation.cc:407] Connection header received
0929 02:23:57.868338 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:57.868342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:57.868407 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:57.868493 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:57.869549 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.870235 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.871885 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.872309 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.872411 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.875814 (+ 3403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:57.875819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:57.875823 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:57.875855 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:57.878283 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.879008 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.879011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.879013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.879124 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:57.879481 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:57.879484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:57.879486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:57.879740 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:57.879854 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:57.880300 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 02:23:57.880464 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:23:57.880882 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.867746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44076)
0929 02:23:57.868022 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.868044 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:57.868191 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:57.868559 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:57.868564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:57.869147 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:57.869370 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.869383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.870368 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.870373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.871733 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:57.871748 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:57.872453 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:57.872459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:57.872666 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:57.873259 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:57.873281 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:57.875641 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:57.878415 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.878421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.878424 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.878899 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.879219 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:57.879224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:57.879228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:57.879391 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:57.879849 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:57.879854 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:57.880044 (+ 190us) client_negotiation.cc:769] Sending connection context
0929 02:23:57.880233 (+ 189us) client_negotiation.cc:240] Negotiation successful
0929 02:23:57.880512 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260929 02:23:57.881953 30306 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:23:57.883033 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.882584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52750)
0929 02:23:57.882759 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.882854 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260929 02:23:57.883913 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.883574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52756)
0929 02:23:57.883702 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.883799 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260929 02:23:57.884104 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:57.913969 30380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 3
I20260929 02:23:57.915248 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.914673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52764)
0929 02:23:57.914916 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.915045 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260929 02:23:57.916373 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.915867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52772)
0929 02:23:57.916111 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.916215 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260929 02:23:57.916644 30380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:57.935328 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 1
I20260929 02:23:57.936470 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.935981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52786)
0929 02:23:57.936176 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.936279 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260929 02:23:57.937417 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:57.937080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52798)
0929 02:23:57.937213 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:57.937292 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:23:57.937633 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:58.005870 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 2
I20260929 02:23:58.007125 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.006548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52812)
0929 02:23:58.006790 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.006906 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260929 02:23:58.008158 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.007767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52822)
0929 02:23:58.007916 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.008012 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 02:23:58.008358 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
W20260929 02:23:58.031184 30387 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:23:58.031495 30387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:23:58.031564 30387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:23:58.031644 30387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:23:58.038762 30387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:23:58.038851 30387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:23:58.038905 30387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:23:58.038952 30387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:23:58.039009 30387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:23:58.048671 30387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:36349,127.7.170.61:33517,127.7.170.60:37385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:37385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.170.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:23:58.049913 30387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:23:58.051129 30387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260929 02:23:58.059842 30380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 4
I20260929 02:23:58.061043 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.060463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52832)
0929 02:23:58.060719 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.060829 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260929 02:23:58.062175 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.061734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52846)
0929 02:23:58.061962 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.062045 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260929 02:23:58.062407 30380 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
W20260929 02:23:58.062996 30392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:58.063050 30393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:23:58.063094 30395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:23:58.064121 30387 server_base.cc:1034] running on GCE node
I20260929 02:23:58.064880 30387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:23:58.066265 30387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:23:58.067484 30387 hybrid_clock.cc:648] HybridClock initialized: now 1790648638067413 us; error 44 us; skew 500 ppm
I20260929 02:23:58.067800 30387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:23:58.070817 30387 init.cc:377] Logged in from keytab as kudu/127.7.170.60@KRBTEST.COM (short username kudu)
I20260929 02:23:58.072175 30387 webserver.cc:460] Webserver started at http://127.7.170.60:45511/ using document root <none> and password file <none>
I20260929 02:23:58.072835 30387 fs_manager.cc:359] Metadata directory not provided
I20260929 02:23:58.072950 30387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:23:58.073242 30387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:23:58.076093 30387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data/instance:
uuid: "994ef527cb7d4261b065dd26d717f1e4"
format_stamp: "Formatted at 2026-09-29 02:23:58 on dist-test-slave-8cs2"
server_key: "907e54b61980f92b1abea452fb122e98"
server_key_iv: "32d18e2427974d819b1f834b6fdb4365"
server_key_version: "encryptionkey@0"
I20260929 02:23:58.076795 30387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal/instance:
uuid: "994ef527cb7d4261b065dd26d717f1e4"
format_stamp: "Formatted at 2026-09-29 02:23:58 on dist-test-slave-8cs2"
server_key: "907e54b61980f92b1abea452fb122e98"
server_key_iv: "32d18e2427974d819b1f834b6fdb4365"
server_key_version: "encryptionkey@0"
I20260929 02:23:58.081976 30387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260929 02:23:58.085471 30402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:23:58.086916 30387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260929 02:23:58.087078 30387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "994ef527cb7d4261b065dd26d717f1e4"
format_stamp: "Formatted at 2026-09-29 02:23:58 on dist-test-slave-8cs2"
server_key: "907e54b61980f92b1abea452fb122e98"
server_key_iv: "32d18e2427974d819b1f834b6fdb4365"
server_key_version: "encryptionkey@0"
I20260929 02:23:58.087280 30387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:23:58.107174 30387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:23:58.111274 30387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:23:58.111521 30387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:23:58.114558 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 3
I20260929 02:23:58.115648 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.115164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52862)
0929 02:23:58.115358 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.115477 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260929 02:23:58.116747 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.116252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52866)
0929 02:23:58.116506 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.116592 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260929 02:23:58.117048 30306 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:37385: Network error: Client connection negotiation failed: client connection to 127.7.170.60:37385: connect: Connection refused (error 111)
I20260929 02:23:58.128767 30387 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.60:37385
I20260929 02:23:58.128782 30453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.60:37385 every 8 connection(s)
I20260929 02:23:58.130187 30387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
I20260929 02:23:58.133708 30454 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:23:58.136212 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30387
I20260929 02:23:58.136376 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790648296482495-7848-0/minicluster-data/master-2/wal/instance
I20260929 02:23:58.136791 7848 external_mini_cluster.cc:1428] Setting key ba547e9c33aad30130948e78d13804b2
I20260929 02:23:58.140501 30454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:58.157524 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.143480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42196 (local address 127.7.170.62:36349)
0929 02:23:58.143663 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.143669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.144364 (+ 695us) server_negotiation.cc:407] Connection header received
0929 02:23:58.144520 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.144523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.144580 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.144652 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.145968 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.146957 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.148671 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.149139 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.149246 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.152440 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.152446 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.152451 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.152492 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.155003 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.155605 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.155610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.155614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.155737 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.156121 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.156126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.156129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.156367 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.156504 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.156983 (+ 479us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.157160 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260929 02:23:58.158195 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.143486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:36349 (local address 127.0.0.1:42196)
0929 02:23:58.144228 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.144269 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:58.144413 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:58.144734 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:58.144741 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:58.145537 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:58.145801 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.145810 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.147092 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.147098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.148547 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.148557 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.149348 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.149356 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.149497 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.150091 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:58.150112 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:58.152260 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:58.155118 (+ 2858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.155132 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.155139 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.155503 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.155835 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.155841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.155844 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.156023 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.156500 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:58.156508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:58.156683 (+ 175us) client_negotiation.cc:769] Sending connection context
0929 02:23:58.156846 (+ 163us) client_negotiation.cc:240] Negotiation successful
0929 02:23:58.157302 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":163,"spinlock_wait_cycles":28800,"thread_start_us":355,"threads_started":1}
I20260929 02:23:58.158212 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.138528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52868 (local address 127.7.170.60:37385)
0929 02:23:58.139246 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.139258 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.139296 (+ 38us) server_negotiation.cc:407] Connection header received
0929 02:23:58.140872 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.140901 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.141311 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.141615 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.143368 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.145296 (+ 1928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.146936 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.147934 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.148101 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.151986 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.151993 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.152005 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.152032 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.154464 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.155063 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.155069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.155075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.155228 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.155558 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.155561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.155563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.156042 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.156313 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.156716 (+ 403us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.157175 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":515,"thread_start_us":328,"threads_started":1}
I20260929 02:23:58.159646 30454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:23:58.176774 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.160331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:33517 (local address 127.0.0.1:44090)
0929 02:23:58.160496 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.160512 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:58.160619 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:58.160921 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:58.160945 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:58.161231 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:58.161363 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.161371 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.162273 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.162279 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.163130 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.163141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.163791 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.163797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.163963 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.164432 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:58.164446 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:58.169961 (+ 5515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:58.174329 (+ 4368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.174334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.174338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.174736 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.175177 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.175182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.175185 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.175344 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.175893 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:58.175898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:58.176049 (+ 151us) client_negotiation.cc:769] Sending connection context
0929 02:23:58.176185 (+ 136us) client_negotiation.cc:240] Negotiation successful
0929 02:23:58.176440 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260929 02:23:58.176856 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.160318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44090 (local address 127.7.170.61:33517)
0929 02:23:58.160492 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.160497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.160574 (+ 77us) server_negotiation.cc:407] Connection header received
0929 02:23:58.160732 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.160736 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.160798 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.160871 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.161499 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.162148 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.163256 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.163667 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.163772 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.170822 (+ 7050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.170830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.170835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.170878 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.174189 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.174906 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.174911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.174915 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.175074 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.175436 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.175441 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.175444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.175751 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.175886 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.176310 (+ 424us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.176507 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260929 02:23:58.177126 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.161396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42198 (local address 127.7.170.62:36349)
0929 02:23:58.161550 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.161554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.161568 (+ 14us) server_negotiation.cc:407] Connection header received
0929 02:23:58.161601 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.161604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.161648 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.161719 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.162754 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.163440 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.164630 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.165081 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.165182 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.170927 (+ 5745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.170933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.170937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.170972 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.174439 (+ 3467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.175083 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.175087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.175089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.175233 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.175594 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.175598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.175600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.175853 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.176114 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.176312 (+ 198us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.176503 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260929 02:23:58.177776 30454 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648638, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:23:58.190056 30458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.178415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52884)
0929 02:23:58.178567 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.178583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:58.178664 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:58.179078 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:58.179083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:58.179422 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:58.179548 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.179556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.180402 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.180407 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.181258 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.181269 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.181919 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.181924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.182100 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.182585 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:58.182600 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:58.184393 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:58.187404 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.187408 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.187410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.187863 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.188336 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.188340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.188342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.188509 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.189056 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:58.189061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:58.189223 (+ 162us) client_negotiation.cc:769] Sending connection context
0929 02:23:58.189396 (+ 173us) client_negotiation.cc:240] Negotiation successful
0929 02:23:58.189684 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260929 02:23:58.190115 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.178723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52884 (local address 127.7.170.60:37385)
0929 02:23:58.178878 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.178882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.178893 (+ 11us) server_negotiation.cc:407] Connection header received
0929 02:23:58.178925 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.178929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.178974 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.179047 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.179662 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.180295 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.181378 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.181770 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.181902 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.184547 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.184551 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.184554 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.184586 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.187284 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.188040 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.188044 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.188046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.188177 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.188619 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.188629 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.188631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.188905 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.189061 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.189499 (+ 438us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.189849 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":143,"server-negotiator.queue_time_us":47}
I20260929 02:23:58.192596 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.178446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44106 (local address 127.7.170.61:33517)
0929 02:23:58.178653 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.178658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.178945 (+ 287us) server_negotiation.cc:407] Connection header received
0929 02:23:58.179073 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.179078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.179135 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.179207 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.180279 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.180981 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.182086 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.182621 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.182727 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.186884 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.186889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.186892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.186923 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.189932 (+ 3009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.190611 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.190617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.190620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.190748 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.191153 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.191156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.191159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.191418 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.191658 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.192074 (+ 416us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.192294 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260929 02:23:58.195578 30454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: Bootstrap starting.
I20260929 02:23:58.196853 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:23:58.198553 30454 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:58.199523 30454 log.cc:824] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:58.199342 30281 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "af447091f25d432d9f54abda8749ba46"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42198
I20260929 02:23:58.202028 30454 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: No bootstrap required, opened a new log
I20260929 02:23:58.208773 30454 raft_consensus.cc:348] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.209137 30454 raft_consensus.cc:729] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 994ef527cb7d4261b065dd26d717f1e4, State: Initialized, Role: FOLLOWER
I20260929 02:23:58.209537 30454 consensus_queue.cc:260] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.210415 30460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.210795 30460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.211406 30454 sys_catalog.cc:564] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: configured and running, proceeding with master startup.
W20260929 02:23:58.220044 30471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:23:58.220153 30471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:23:58.218258 30387 master_runner.cc:186] Error getting master registration for 127.7.170.62:36349: OK, instance_id { permanent_uuid: "af447091f25d432d9f54abda8749ba46" instance_seqno: 1790648637161376 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:23:58.229069 30387 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:23:58.294262 30306 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 4
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
I20260929 02:23:58.309314 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.295085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.7.170.60:37385)
0929 02:23:58.295312 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.295318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.295345 (+ 27us) server_negotiation.cc:407] Connection header received
0929 02:23:58.295484 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.295490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.295585 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.295673 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.296581 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.297380 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.298571 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.299022 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.299127 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.303004 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.303010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.303018 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.303057 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.306397 (+ 3340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.307150 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.307156 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.307159 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.307315 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.307684 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.307688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.307692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.308138 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.308290 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.308694 (+ 404us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.308918 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260929 02:23:58.309319 30309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.295023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52886)
0929 02:23:58.295203 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.295224 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:58.295345 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:58.295733 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:58.295738 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:58.296264 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:58.296442 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.296452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.297519 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.297525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.298435 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.298448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.299160 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.299166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.299363 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.299984 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:58.300008 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:58.302809 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:58.306534 (+ 3725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.306540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.306544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.307039 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.307416 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.307421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.307425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.307581 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.308259 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:58.308264 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:58.308453 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 02:23:58.308600 (+ 147us) client_negotiation.cc:240] Negotiation successful
0929 02:23:58.308893 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 02:23:58.315184 30306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: Bootstrap starting.
I20260929 02:23:58.318284 30306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:58.319517 30306 log.cc:824] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:58.322173 30306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: No bootstrap required, opened a new log
I20260929 02:23:58.329344 30306 raft_consensus.cc:348] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.329725 30306 raft_consensus.cc:729] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af447091f25d432d9f54abda8749ba46, State: Initialized, Role: FOLLOWER
I20260929 02:23:58.330206 30306 consensus_queue.cc:260] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.331159 30474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.331507 30474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.332232 30306 sys_catalog.cc:564] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:58.332379 30281 raft_consensus.cc:482] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 02:23:58.332574 30281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:58.335979 30281 raft_consensus.cc:504] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.338404 30429 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af447091f25d432d9f54abda8749ba46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "994ef527cb7d4261b065dd26d717f1e4"
I20260929 02:23:58.338871 30429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:58.342231 30429 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af447091f25d432d9f54abda8749ba46 in term 1.
I20260929 02:23:58.342598 30281 leader_election.cc:288] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [CANDIDATE]: Term 1 election: Requested vote from peers 23a8c69c96154ec7ac2b2553b2341f80 (127.7.170.61:33517), 994ef527cb7d4261b065dd26d717f1e4 (127.7.170.60:37385)
I20260929 02:23:58.343073 30255 leader_election.cc:302] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 994ef527cb7d4261b065dd26d717f1e4, af447091f25d432d9f54abda8749ba46; no voters:
I20260929 02:23:58.343547 30474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 1 FOLLOWER]: Leader election won for term 1
I20260929 02:23:58.345606 30474 raft_consensus.cc:686] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [term 1 LEADER]: Becoming Leader. State: Replica: af447091f25d432d9f54abda8749ba46, State: Running, Role: LEADER
I20260929 02:23:58.346196 30474 consensus_queue.cc:237] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.357184 30482 sys_catalog.cc:455] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af447091f25d432d9f54abda8749ba46. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.357659 30482 sys_catalog.cc:458] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:58.361665 30380 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } attempt: 5
W20260929 02:23:58.363639 30408 tablet.cc:2369] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 29 02:23:58 dist-test-slave-8cs2 krb5kdc[30226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648637, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
I20260929 02:23:58.380366 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.362325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:37385 (local address 127.0.0.1:52890)
0929 02:23:58.362724 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0929 02:23:58.362757 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0929 02:23:58.362898 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:23:58.363820 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:23:58.363827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:23:58.364366 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:23:58.364599 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.364613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.366193 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.366198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.367680 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:23:58.367696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.369085 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.369092 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:23:58.369346 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.370064 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:23:58.370090 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:23:58.372917 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:23:58.377389 (+ 4472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.377395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.377399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.377947 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.378430 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:23:58.378435 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:23:58.378440 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:23:58.378617 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:23:58.379305 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:23:58.379311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:23:58.379494 (+ 183us) client_negotiation.cc:769] Sending connection context
0929 02:23:58.379638 (+ 144us) client_negotiation.cc:240] Negotiation successful
0929 02:23:58.379946 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267}
I20260929 02:23:58.381425 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.363130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52890 (local address 127.7.170.60:37385)
0929 02:23:58.363315 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.363322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.363341 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:23:58.363416 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.363421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.363523 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.363629 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.364789 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.366015 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.367858 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.368578 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.368740 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.373160 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.373168 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.373173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.373223 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.377198 (+ 3975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.378096 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.378102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.378106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.378284 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.378744 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.378749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.378753 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.379105 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.379245 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.380754 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.381011 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260929 02:23:58.387676 30380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: Bootstrap starting.
I20260929 02:23:58.392189 30380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: Neither blocks nor log segments found. Creating new log.
I20260929 02:23:58.393676 30380 log.cc:824] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: Log is configured to *not* fsync() on all Append() calls
I20260929 02:23:58.395218 30487 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:23:58.400460 30380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: No bootstrap required, opened a new log
W20260929 02:23:58.405576 30260 tablet.cc:2369] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:23:58.417325 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.391279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44108 (local address 127.7.170.61:33517)
0929 02:23:58.391455 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.391462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.397599 (+ 6137us) server_negotiation.cc:407] Connection header received
0929 02:23:58.397820 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.397826 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.397899 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.397963 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.400490 (+ 2527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.401674 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.403830 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.404468 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.404561 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.407911 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.407918 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.407923 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.407965 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.411460 (+ 3495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.415029 (+ 3569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.415034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.415038 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.415190 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.415655 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.415660 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.415664 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.415993 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.416216 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.416737 (+ 521us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.416975 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260929 02:23:58.422060 30487 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:23:58.422737 30457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.369939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.7.170.60:37385)
0929 02:23:58.394483 (+ 24544us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.394490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.396796 (+ 2306us) server_negotiation.cc:407] Connection header received
0929 02:23:58.397058 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.397064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.397232 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.397420 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.401183 (+ 3763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.402954 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.406041 (+ 3087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.407117 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.407270 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.412531 (+ 5261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.412539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.412544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.412606 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.417805 (+ 5199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.419017 (+ 1212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.419043 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.419049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.419256 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.419902 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.419908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.419911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.420443 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.420776 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.421619 (+ 843us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.422020 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24383,"thread_start_us":25054,"threads_started":1}
I20260929 02:23:58.427229 30498 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:23:58.434516 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:58.405218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42210 (local address 127.7.170.62:36349)
0929 02:23:58.405501 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.405506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.409434 (+ 3928us) server_negotiation.cc:407] Connection header received
0929 02:23:58.409696 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.409702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.409786 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.409872 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.411650 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.413057 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.421078 (+ 8021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:58.421745 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:58.421894 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:58.424779 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:23:58.424786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:23:58.424791 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:23:58.424852 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:23:58.429537 (+ 4685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.430599 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.430605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.430609 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.430776 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:23:58.431348 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:23:58.431354 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:23:58.431387 (+ 33us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:23:58.431994 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:23:58.432388 (+ 394us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:58.433300 (+ 912us) server_negotiation.cc:299] Negotiation successful
0929 02:23:58.433781 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260929 02:23:58.437427 30380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.437829 30380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23a8c69c96154ec7ac2b2553b2341f80, State: Initialized, Role: FOLLOWER
I20260929 02:23:58.438345 30380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } }
I20260929 02:23:58.430935 30239 master_runner.cc:186] Error getting master registration for 127.7.170.61:33517: OK, instance_id { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" instance_seqno: 1790648637641121 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:23:58.448226 30239 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:23:58.448809 30500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.449285 30500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.454726 30380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:23:58.454792 30355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af447091f25d432d9f54abda8749ba46" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "23a8c69c96154ec7ac2b2553b2341f80"
I20260929 02:23:58.455266 30355 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:23:58.462332 30355 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af447091f25d432d9f54abda8749ba46 in term 1.
I20260929 02:23:58.474761 30355 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [term 1 FOLLOWER]: Refusing update from remote peer af447091f25d432d9f54abda8749ba46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:58.477347 30429 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [term 1 FOLLOWER]: Refusing update from remote peer af447091f25d432d9f54abda8749ba46: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:23:58.481599 30474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:58.482383 30474 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [LEADER]: Connected to new peer: Peer: permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:23:58.503790 30460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af447091f25d432d9f54abda8749ba46. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.504170 30460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.508517 30482 sys_catalog.cc:455] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.508805 30482 sys_catalog.cc:458] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:58.510118 30500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af447091f25d432d9f54abda8749ba46. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.510416 30500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.511665 30474 sys_catalog.cc:455] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.511968 30474 sys_catalog.cc:458] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46 [sys.catalog]: This master's current role is: LEADER
I20260929 02:23:58.512748 30506 mvcc.cc:205] Tried to move back new op lower bound from 7334496823145447424 to 7334496822673715200. Current Snapshot: MvccSnapshot[applied={T|T < 7334496823145447424}]
I20260929 02:23:58.514478 30460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.514744 30460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.521998 30487 catalog_manager.cc:1347] Generated new cluster ID: 36f8f1135eba44a491fdcc04b62114d6
I20260929 02:23:58.522118 30487 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:23:58.522809 30508 mvcc.cc:205] Tried to move back new op lower bound from 7334496823145447424 to 7334496822673715200. Current Snapshot: MvccSnapshot[applied={T|T < 7334496823145447424}]
I20260929 02:23:58.526636 30500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af447091f25d432d9f54abda8749ba46" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af447091f25d432d9f54abda8749ba46" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 36349 } } peers { permanent_uuid: "23a8c69c96154ec7ac2b2553b2341f80" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 33517 } } peers { permanent_uuid: "994ef527cb7d4261b065dd26d717f1e4" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 37385 } } }
I20260929 02:23:58.526907 30500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:23:58.541337 30518 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:23:58.560117 30487 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:23:58.562201 30487 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:23:58.584537 30487 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af447091f25d432d9f54abda8749ba46: Generated new TSK 0
I20260929 02:23:58.585813 30487 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:23:58.688071 30310 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.678471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.7.170.62:36349)
0929 02:23:58.678670 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.678676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.685973 (+ 7297us) server_negotiation.cc:407] Connection header received
0929 02:23:58.686156 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.686162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.686267 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.686356 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.687785 (+ 1429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42212: BlockingRecv error: recv got EOF from 127.0.0.1:42212 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:23:58.688271 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.684802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44110 (local address 127.7.170.61:33517)
0929 02:23:58.685017 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.685024 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.686770 (+ 1746us) server_negotiation.cc:407] Connection header received
0929 02:23:58.686932 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.686937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.687030 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.687127 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.688035 (+ 908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44110: BlockingRecv error: recv got EOF from 127.0.0.1:44110 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 02:23:58.689666 30457 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.678673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.7.170.60:37385)
0929 02:23:58.678808 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.678813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.685654 (+ 6841us) server_negotiation.cc:407] Connection header received
0929 02:23:58.686020 (+ 366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.686027 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.686118 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.686189 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.689388 (+ 3199us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52896: BlockingRecv error: recv got EOF from 127.0.0.1:52896 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260929 02:23:58.706712 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.704416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44126 (local address 127.7.170.61:33517)
0929 02:23:58.704599 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.704606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.705127 (+ 521us) server_negotiation.cc:407] Connection header received
0929 02:23:58.705340 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.705346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.705474 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.705580 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.706442 (+ 862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44126: BlockingRecv error: recv got EOF from 127.0.0.1:44126 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:23:58.716719 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.715059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44134 (local address 127.7.170.61:33517)
0929 02:23:58.715220 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.715224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.715408 (+ 184us) server_negotiation.cc:407] Connection header received
0929 02:23:58.715568 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.715572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.715640 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.715723 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.716520 (+ 797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44134: BlockingRecv error: recv got EOF from 127.0.0.1:44134 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260929 02:23:58.723610 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.721872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44146 (local address 127.7.170.61:33517)
0929 02:23:58.722041 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.722045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.722307 (+ 262us) server_negotiation.cc:407] Connection header received
0929 02:23:58.722481 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.722485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.722561 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.722640 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.723394 (+ 754us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44146: BlockingRecv error: recv got EOF from 127.0.0.1:44146 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 02:23:58.732209 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.730512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44162 (local address 127.7.170.61:33517)
0929 02:23:58.730684 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.730688 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.730932 (+ 244us) server_negotiation.cc:407] Connection header received
0929 02:23:58.731089 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.731092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.731170 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.731250 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.731988 (+ 738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44162: BlockingRecv error: recv got EOF from 127.0.0.1:44162 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260929 02:23:58.745867 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.743865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44176 (local address 127.7.170.61:33517)
0929 02:23:58.744070 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.744075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.744376 (+ 301us) server_negotiation.cc:407] Connection header received
0929 02:23:58.744564 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.744567 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.744637 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.744715 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.745599 (+ 884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44176: BlockingRecv error: recv got EOF from 127.0.0.1:44176 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260929 02:23:58.767431 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.765307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44178 (local address 127.7.170.61:33517)
0929 02:23:58.765520 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.765527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.766077 (+ 550us) server_negotiation.cc:407] Connection header received
0929 02:23:58.766248 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.766252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.766325 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.766425 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.767208 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44178: BlockingRecv error: recv got EOF from 127.0.0.1:44178 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260929 02:23:58.805017 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.802973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44194 (local address 127.7.170.61:33517)
0929 02:23:58.803207 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.803213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.803556 (+ 343us) server_negotiation.cc:407] Connection header received
0929 02:23:58.803720 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.803726 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.803840 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.803979 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.804705 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44194: BlockingRecv error: recv got EOF from 127.0.0.1:44194 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260929 02:23:58.887980 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:58.885874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44210 (local address 127.7.170.61:33517)
0929 02:23:58.886080 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:58.886087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:58.886596 (+ 509us) server_negotiation.cc:407] Connection header received
0929 02:23:58.886783 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:58.886787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:58.886859 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:58.886969 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:58.887742 (+ 773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44210: BlockingRecv error: recv got EOF from 127.0.0.1:44210 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260929 02:23:59.021499 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:59.019650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44220 (local address 127.7.170.61:33517)
0929 02:23:59.019837 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.019843 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.020145 (+ 302us) server_negotiation.cc:407] Connection header received
0929 02:23:59.020351 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.020355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.020429 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.020509 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:59.021248 (+ 739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44220: BlockingRecv error: recv got EOF from 127.0.0.1:44220 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:23:59.222131 30471 catalog_manager.cc:1259] Loaded cluster ID: 36f8f1135eba44a491fdcc04b62114d6
I20260929 02:23:59.222234 30471 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: loading cluster ID for follower catalog manager: success
I20260929 02:23:59.224658 30471 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: acquiring CA information for follower catalog manager: success
I20260929 02:23:59.226711 30471 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 994ef527cb7d4261b065dd26d717f1e4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:23:59.282925 30384 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:23:59.281176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44228 (local address 127.7.170.61:33517)
0929 02:23:59.281428 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.281434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.281543 (+ 109us) server_negotiation.cc:407] Connection header received
0929 02:23:59.281707 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.281711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.281783 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.281871 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:23:59.282705 (+ 834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44228: BlockingRecv error: recv got EOF from 127.0.0.1:44228 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260929 02:23:59.546640 30518 catalog_manager.cc:1259] Loaded cluster ID: 36f8f1135eba44a491fdcc04b62114d6
I20260929 02:23:59.546772 30518 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: loading cluster ID for follower catalog manager: success
I20260929 02:23:59.549649 30518 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: acquiring CA information for follower catalog manager: success
I20260929 02:23:59.551749 30518 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 23a8c69c96154ec7ac2b2553b2341f80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:23:59.810572 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:59.802848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.7.170.61:33517)
0929 02:23:59.803782 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.803786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.803806 (+ 20us) server_negotiation.cc:407] Connection header received
0929 02:23:59.803872 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.803876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.804057 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.804177 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:59.804740 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.805799 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.807787 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.808255 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.808351 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:59.809058 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:59.809831 (+ 773us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:59.809962 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:59.810117 (+ 155us) server_negotiation.cc:299] Negotiation successful
0929 02:23:59.810196 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":652,"threads_started":1}
I20260929 02:23:59.825438 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:59.818363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52900 (local address 127.7.170.60:37385)
0929 02:23:59.819968 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.819973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.819989 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:23:59.820100 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.820105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.820284 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.820452 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:59.820866 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.821687 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.823042 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.823455 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.823575 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:59.824085 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:59.824757 (+ 672us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:59.824921 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:59.825018 (+ 97us) server_negotiation.cc:299] Negotiation successful
0929 02:23:59.825114 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1492,"thread_start_us":1390,"threads_started":1}
I20260929 02:23:59.849056 30591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:59.834046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.7.170.60:37385)
0929 02:23:59.834331 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.834337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.842094 (+ 7757us) server_negotiation.cc:407] Connection header received
0929 02:23:59.842267 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.842273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.842373 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.842469 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:59.843281 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.844446 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.846677 (+ 2231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.847369 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.847474 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:59.848122 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:59.848268 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:59.848333 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:59.848602 (+ 269us) server_negotiation.cc:299] Negotiation successful
0929 02:23:59.848701 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260929 02:23:59.850687 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:59.833800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.7.170.62:36349)
0929 02:23:59.840788 (+ 6988us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.840795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.840820 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:23:59.841031 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.841038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.841336 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.842463 (+ 1127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:59.843449 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.844721 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.847741 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.848376 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.848499 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:59.849185 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:59.849732 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:59.849846 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:59.850280 (+ 434us) server_negotiation.cc:299] Negotiation successful
0929 02:23:59.850374 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6805,"thread_start_us":6603,"threads_started":1}
I20260929 02:23:59.855046 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:23:59.833516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.7.170.61:33517)
0929 02:23:59.833778 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0929 02:23:59.833785 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:23:59.847579 (+ 13794us) server_negotiation.cc:407] Connection header received
0929 02:23:59.847738 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:23:59.847743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:23:59.847848 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:23:59.847935 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:23:59.848716 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.849828 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.852869 (+ 3041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:23:59.853591 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:23:59.853731 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:23:59.854187 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:23:59.854332 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:23:59.854431 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0929 02:23:59.854610 (+ 179us) server_negotiation.cc:299] Negotiation successful
0929 02:23:59.854707 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260929 02:23:59.856190 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30239
I20260929 02:23:59.857510 30301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:23:59.981062 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30239
I20260929 02:24:00.003206 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30313
I20260929 02:24:00.004592 30375 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:24:00.120210 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30313
I20260929 02:24:00.140177 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30387
I20260929 02:24:00.141539 30449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:24:00.259656 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30387
2026-09-29T02:24:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4833 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 29 02:24:00 dist-test-slave-8cs2 krb5kdc[30606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 29 02:24:00 dist-test-slave-8cs2 krb5kdc[30606](info): set up 2 sockets
Sep 29 02:24:00 dist-test-slave-8cs2 krb5kdc[30606](info): commencing operation
krb5kdc: starting...
W20260929 02:24:01.305577 7848 mini_kdc.cc:121] Time spent starting KDC: real 1.005s user 0.006s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/test-user.keytab.
Sep 29 02:24:01 dist-test-slave-8cs2 krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-29T02:24:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-29T02:24:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.170.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
Entry for principal kudu/127.7.170.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab.
I20260929 02:24:01.574644 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38737
--webserver_interface=127.7.170.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260929 02:24:01.910794 30619 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:24:01.911134 30619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:24:01.911231 30619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:24:01.911351 30619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:24:01.918917 30619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:24:01.919005 30619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:24:01.919075 30619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:24:01.919147 30619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:24:01.919229 30619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:24:01.929167 30619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.62.keytab
--principal=kudu/127.7.170.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.62:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.170.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:24:01.930470 30619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:24:01.931800 30619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:24:01.943568 30625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:01.943542 30627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:01.943797 30624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:24:01.944219 30619 server_base.cc:1034] running on GCE node
I20260929 02:24:01.945083 30619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:24:01.946535 30619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:24:01.947742 30619 hybrid_clock.cc:648] HybridClock initialized: now 1790648641947675 us; error 43 us; skew 500 ppm
I20260929 02:24:01.948042 30619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:24:01 dist-test-slave-8cs2 krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:24:01.951191 30619 init.cc:377] Logged in from keytab as kudu/127.7.170.62@KRBTEST.COM (short username kudu)
I20260929 02:24:01.952643 30619 webserver.cc:460] Webserver started at http://127.7.170.62:38101/ using document root <none> and password file <none>
I20260929 02:24:01.953342 30619 fs_manager.cc:359] Metadata directory not provided
I20260929 02:24:01.953439 30619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:24:01.953747 30619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:24:01.956773 30619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data/instance:
uuid: "b805a9561f2f4318b851366f819d76a1"
format_stamp: "Formatted at 2026-09-29 02:24:01 on dist-test-slave-8cs2"
server_key: "659bcf97755af48ddfe485c339e72b38"
server_key_iv: "4ace64744cea3e31cc8bb8be54a25e6f"
server_key_version: "encryptionkey@0"
I20260929 02:24:01.957628 30619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance:
uuid: "b805a9561f2f4318b851366f819d76a1"
format_stamp: "Formatted at 2026-09-29 02:24:01 on dist-test-slave-8cs2"
server_key: "659bcf97755af48ddfe485c339e72b38"
server_key_iv: "4ace64744cea3e31cc8bb8be54a25e6f"
server_key_version: "encryptionkey@0"
I20260929 02:24:01.962793 30619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260929 02:24:01.966200 30634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:24:01.967533 30619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260929 02:24:01.967691 30619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal
uuid: "b805a9561f2f4318b851366f819d76a1"
format_stamp: "Formatted at 2026-09-29 02:24:01 on dist-test-slave-8cs2"
server_key: "659bcf97755af48ddfe485c339e72b38"
server_key_iv: "4ace64744cea3e31cc8bb8be54a25e6f"
server_key_version: "encryptionkey@0"
I20260929 02:24:01.967891 30619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:24:01.993501 30619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:24:01.997639 30619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:24:01.997895 30619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:24:02.015622 30619 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.62:38737
I20260929 02:24:02.015636 30685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.62:38737 every 8 connection(s)
I20260929 02:24:02.016963 30619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/data/info.pb
I20260929 02:24:02.020586 30686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:24:02.021842 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30619
I20260929 02:24:02.022012 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-0/wal/instance
I20260929 02:24:02.022475 7848 external_mini_cluster.cc:1428] Setting key 4fb1e5bd5f70dea7f5ceafe913cd0112
I20260929 02:24:02.027114 30686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.170.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.61@KRBTEST.COM" created.
I20260929 02:24:02.048756 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.030327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.7.170.62:38737)
0929 02:24:02.031114 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:02.031131 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:02.031561 (+ 430us) server_negotiation.cc:407] Connection header received
0929 02:24:02.033585 (+ 2024us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:02.033591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:02.033832 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:02.034010 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:02.035240 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.036439 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.037799 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.038341 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.038503 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.041674 (+ 3171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:02.041682 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:02.041694 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:02.041727 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:02.044604 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.045338 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.045345 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.045347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.045498 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.045910 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.045917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.045921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.046469 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:02.046619 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:02.047278 (+ 659us) server_negotiation.cc:299] Negotiation successful
0929 02:24:02.047523 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":574,"thread_start_us":362,"threads_started":1}
I20260929 02:24:02.048794 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.030312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38737 (local address 127.0.0.1:42786)
0929 02:24:02.031411 (+ 1099us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.031450 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:02.033228 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:02.034040 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:02.034046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:02.034717 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:02.035044 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.035061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.036572 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.036576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.037678 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.037689 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.038557 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.038565 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.038780 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.039365 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:02.039387 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:02.041469 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:02.044745 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.044756 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.044769 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.045210 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.045599 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.045604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.045607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.045753 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.046645 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:02.046654 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:02.046923 (+ 269us) client_negotiation.cc:769] Sending connection context
0929 02:24:02.047134 (+ 211us) client_negotiation.cc:240] Negotiation successful
0929 02:24:02.047654 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"mutex_wait_us":147,"thread_start_us":683,"threads_started":1}
I20260929 02:24:02.051559 30686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:24:02.052584 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.052235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47316)
0929 02:24:02.052383 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.052484 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260929 02:24:02.052846 30635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
I20260929 02:24:02.054337 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.053961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47322)
0929 02:24:02.054125 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.054198 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 02:24:02.054548 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36379: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
Entry for principal kudu/127.7.170.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab.
I20260929 02:24:02.078660 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:36379
--webserver_interface=127.7.170.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:24:02.103907 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } attempt: 1
I20260929 02:24:02.105114 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.104630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47336)
0929 02:24:02.104817 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.104982 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260929 02:24:02.106061 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.105709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47340)
0929 02:24:02.105841 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.105912 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260929 02:24:02.106216 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36379: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
I20260929 02:24:02.174454 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } attempt: 2
I20260929 02:24:02.175576 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.175131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47346)
0929 02:24:02.175336 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.175439 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260929 02:24:02.176608 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.176255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47358)
0929 02:24:02.176392 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.176475 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260929 02:24:02.176808 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36379: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
I20260929 02:24:02.268064 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } attempt: 3
I20260929 02:24:02.269315 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.268822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47368)
0929 02:24:02.269077 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.269169 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260929 02:24:02.270355 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.270017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47370)
0929 02:24:02.270146 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.270219 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260929 02:24:02.270551 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36379: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
W20260929 02:24:02.398151 30693 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:24:02.398494 30693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:24:02.398563 30693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:24:02.398644 30693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:24:02.405826 30693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:24:02.405925 30693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:24:02.405980 30693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:24:02.406030 30693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:24:02.406095 30693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:24:02.413787 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } attempt: 4
I20260929 02:24:02.414965 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.414425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47382)
0929 02:24:02.414734 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.414824 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260929 02:24:02.416081 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.415659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47388)
0929 02:24:02.415837 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.415944 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260929 02:24:02.416296 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.61:36379: Network error: Client connection negotiation failed: client connection to 127.7.170.61:36379: connect: Connection refused (error 111)
I20260929 02:24:02.416186 30693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.61.keytab
--principal=kudu/127.7.170.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.61:36379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.170.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:24:02.417603 30693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:24:02.419093 30693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:24:02.430400 30698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:02.430546 30699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:02.430593 30701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:24:02.431644 30693 server_base.cc:1034] running on GCE node
I20260929 02:24:02.432431 30693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:24:02.433943 30693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:24:02.435164 30693 hybrid_clock.cc:648] HybridClock initialized: now 1790648642435097 us; error 43 us; skew 500 ppm
I20260929 02:24:02.435485 30693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:24:02.438611 30693 init.cc:377] Logged in from keytab as kudu/127.7.170.61@KRBTEST.COM (short username kudu)
I20260929 02:24:02.440050 30693 webserver.cc:460] Webserver started at http://127.7.170.61:36339/ using document root <none> and password file <none>
I20260929 02:24:02.440680 30693 fs_manager.cc:359] Metadata directory not provided
I20260929 02:24:02.440763 30693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:24:02.441085 30693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:24:02.444083 30693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data/instance:
uuid: "a5767c645d194065ba60e58178a85e11"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "da3d672650187a454ecc5abfcc64b60e"
server_key_iv: "f1cc9931a60c67edcb2a28214b0fd14b"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.444799 30693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal/instance:
uuid: "a5767c645d194065ba60e58178a85e11"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "da3d672650187a454ecc5abfcc64b60e"
server_key_iv: "f1cc9931a60c67edcb2a28214b0fd14b"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.449905 30693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260929 02:24:02.453195 30708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:24:02.454515 30693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260929 02:24:02.454661 30693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal
uuid: "a5767c645d194065ba60e58178a85e11"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "da3d672650187a454ecc5abfcc64b60e"
server_key_iv: "f1cc9931a60c67edcb2a28214b0fd14b"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.454864 30693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:24:02.466560 30693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:24:02.470480 30693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:24:02.470710 30693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:24:02.487326 30693 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.61:36379
I20260929 02:24:02.487346 30759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.61:36379 every 8 connection(s)
I20260929 02:24:02.488626 30693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/data/info.pb
I20260929 02:24:02.492213 30760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:24:02.493602 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30693
I20260929 02:24:02.493774 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-1/wal/instance
I20260929 02:24:02.494168 7848 external_mini_cluster.cc:1428] Setting key f0174d0c7a32506f64e67095e64e9c24
I20260929 02:24:02.498679 30760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:24:02.519308 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.501397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42788 (local address 127.7.170.62:38737)
0929 02:24:02.501604 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:02.501611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:02.502881 (+ 1270us) server_negotiation.cc:407] Connection header received
0929 02:24:02.505066 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:02.505072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:02.505158 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:02.505245 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:02.506780 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.507487 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.509015 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.509584 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.509707 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.513322 (+ 3615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:02.513328 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:02.513331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:02.513367 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:02.515949 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.516867 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.516872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.516874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.517029 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.517493 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.517498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.517500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.517794 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:02.517918 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:02.518894 (+ 976us) server_negotiation.cc:299] Negotiation successful
0929 02:24:02.519075 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
WARNING: no policy specified for kudu/127.7.170.60@KRBTEST.COM; defaulting to no policy
I20260929 02:24:02.520445 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.501885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38737 (local address 127.0.0.1:42788)
0929 02:24:02.502684 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.502732 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:02.504765 (+ 2033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:02.505500 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:02.505508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:02.506062 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:02.506612 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.506629 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.507645 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.507648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.508845 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.508858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.509823 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.509831 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.510077 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.510732 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:02.510755 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:02.513072 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:02.516097 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.516107 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.516126 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.516734 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.517127 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.517132 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.517136 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.517373 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.517929 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:02.517938 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:02.518403 (+ 465us) client_negotiation.cc:769] Sending connection context
0929 02:24:02.518735 (+ 332us) client_negotiation.cc:240] Negotiation successful
0929 02:24:02.519290 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":309,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.170.60@KRBTEST.COM" created.
I20260929 02:24:02.522224 30760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:24:02.537096 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.522919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47394)
0929 02:24:02.523096 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.523114 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:02.523213 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:02.524418 (+ 1205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:02.524422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:02.524764 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:02.524963 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.524975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.526651 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.526655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.527533 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.527544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.528547 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.528552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.528677 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.529261 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:02.529278 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:02.531386 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:02.534650 (+ 3264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.534655 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.534660 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.535104 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.535426 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.535431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.535434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.535603 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.536148 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:02.536153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:02.536295 (+ 142us) client_negotiation.cc:769] Sending connection context
0929 02:24:02.536432 (+ 137us) client_negotiation.cc:240] Negotiation successful
0929 02:24:02.536701 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260929 02:24:02.537170 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.523180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47394 (local address 127.7.170.61:36379)
0929 02:24:02.523844 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:02.523852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:02.523883 (+ 31us) server_negotiation.cc:407] Connection header received
0929 02:24:02.523934 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:02.523942 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:02.524226 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:02.524405 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:02.525126 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.526517 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.527659 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.528382 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.528512 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.531587 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:02.531601 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:02.531618 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:02.531656 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:02.534527 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.535220 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.535226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.535228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.535336 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.535695 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.535698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.535700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.536025 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:02.536174 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:02.536571 (+ 397us) server_negotiation.cc:299] Negotiation successful
0929 02:24:02.536802 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":494,"thread_start_us":355,"threads_started":1}
I20260929 02:24:02.539769 30760 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:24:02.540818 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.540330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51700)
0929 02:24:02.540507 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.540641 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260929 02:24:02.541098 30712 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.542567 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.542207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51710)
0929 02:24:02.542351 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.542441 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260929 02:24:02.542752 30760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
Entry for principal kudu/127.7.170.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab.
I20260929 02:24:02.555792 7848 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
/tmp/dist-test-taskoo3C6V/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:36327
--webserver_interface=127.7.170.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.170.20:43549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260929 02:24:02.592092 30760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 1
I20260929 02:24:02.593393 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.592797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51726)
0929 02:24:02.593078 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.593204 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260929 02:24:02.594412 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.594034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51740)
0929 02:24:02.594182 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.594276 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260929 02:24:02.594619 30760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.662855 30760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 2
I20260929 02:24:02.664096 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.663560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51744)
0929 02:24:02.663786 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.663902 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260929 02:24:02.665161 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.664764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51754)
0929 02:24:02.664899 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.665018 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260929 02:24:02.665422 30760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.715552 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } attempt: 5
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:24:02.729092 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.716403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47410 (local address 127.7.170.61:36379)
0929 02:24:02.716587 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:02.716593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:02.716687 (+ 94us) server_negotiation.cc:407] Connection header received
0929 02:24:02.716887 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:02.716892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:02.716999 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:02.717084 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:02.717936 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.718634 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.719790 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.720202 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.720311 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.723639 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:02.723645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:02.723648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:02.723684 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:02.726410 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.727250 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.727254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.727257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.727384 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:02.727754 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:02.727757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:02.727759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:02.728045 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:02.728148 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:02.728594 (+ 446us) server_negotiation.cc:299] Negotiation successful
0929 02:24:02.728760 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:24:02.729163 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.716354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47410)
0929 02:24:02.716588 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.716608 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:02.716753 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:02.717148 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:02.717153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:02.717564 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:02.717744 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.717754 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.718769 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.718774 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.719630 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:02.719649 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:02.720364 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:02.720370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:02.720574 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:02.721186 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:02.721205 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:02.723427 (+ 2222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:02.726568 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.726574 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.726578 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.727140 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.727487 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:02.727491 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:02.727495 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:02.727668 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:02.728153 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:02.728158 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:02.728342 (+ 184us) client_negotiation.cc:769] Sending connection context
0929 02:24:02.728507 (+ 165us) client_negotiation.cc:240] Negotiation successful
0929 02:24:02.728800 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260929 02:24:02.730180 30686 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:24:02.731205 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.730768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51768)
0929 02:24:02.730946 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.731041 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260929 02:24:02.732236 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.731847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51784)
0929 02:24:02.732038 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.732120 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260929 02:24:02.732431 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.756677 30760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 3
I20260929 02:24:02.757938 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.757468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51794)
0929 02:24:02.757673 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.757768 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260929 02:24:02.759111 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.758674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51802)
0929 02:24:02.758891 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.758993 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260929 02:24:02.759334 30760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.783690 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 1
I20260929 02:24:02.784993 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.784424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51812)
0929 02:24:02.784636 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.784738 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260929 02:24:02.786090 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.785737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51822)
0929 02:24:02.785903 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.785981 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260929 02:24:02.786314 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.854518 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 2
I20260929 02:24:02.855727 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.855274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51828)
0929 02:24:02.855475 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.855564 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260929 02:24:02.856750 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.856411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51832)
0929 02:24:02.856573 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.856644 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260929 02:24:02.856995 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
W20260929 02:24:02.887769 30767 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260929 02:24:02.888120 30767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260929 02:24:02.888214 30767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260929 02:24:02.888327 30767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260929 02:24:02.895671 30767 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260929 02:24:02.895783 30767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260929 02:24:02.895867 30767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260929 02:24:02.895941 30767 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260929 02:24:02.896024 30767 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260929 02:24:02.902551 30760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 4
I20260929 02:24:02.903712 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.903225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51848)
0929 02:24:02.903468 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.903558 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260929 02:24:02.904598 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.904295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51860)
0929 02:24:02.904432 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.904497 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260929 02:24:02.904779 30760 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.906484 30767 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.170.20:43549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.170.62:38737,127.7.170.61:36379,127.7.170.60:36327
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/krb5kdc/kudu_127.7.170.60.keytab
--principal=kudu/127.7.170.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.170.60:36327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.170.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 29 Sep 2026 01:05:47 UTC on ccycloud.quasar-uepxvq.root.comops.site
build id 4679060
ASAN enabled
I20260929 02:24:02.907773 30767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260929 02:24:02.909099 30767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260929 02:24:02.920207 30773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:02.920393 30775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260929 02:24:02.920589 30772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260929 02:24:02.921228 30767 server_base.cc:1034] running on GCE node
I20260929 02:24:02.922016 30767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260929 02:24:02.923385 30767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260929 02:24:02.924605 30767 hybrid_clock.cc:648] HybridClock initialized: now 1790648642924535 us; error 48 us; skew 500 ppm
I20260929 02:24:02.924909 30767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Sep 29 02:24:02 dist-test-slave-8cs2 krb5kdc[30606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260929 02:24:02.927842 30767 init.cc:377] Logged in from keytab as kudu/127.7.170.60@KRBTEST.COM (short username kudu)
I20260929 02:24:02.929219 30767 webserver.cc:460] Webserver started at http://127.7.170.60:39623/ using document root <none> and password file <none>
I20260929 02:24:02.929889 30767 fs_manager.cc:359] Metadata directory not provided
I20260929 02:24:02.929988 30767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260929 02:24:02.930289 30767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260929 02:24:02.933301 30767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data/instance:
uuid: "da33d845d46e418987b2f77428df3c0a"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "7aa81794e62ddeb2a3e157df140c2a09"
server_key_iv: "481436072a919e984bced1afefa9886d"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.934060 30767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal/instance:
uuid: "da33d845d46e418987b2f77428df3c0a"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "7aa81794e62ddeb2a3e157df140c2a09"
server_key_iv: "481436072a919e984bced1afefa9886d"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.939285 30767 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260929 02:24:02.942700 30782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260929 02:24:02.944195 30767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260929 02:24:02.944373 30767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data,/tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal
uuid: "da33d845d46e418987b2f77428df3c0a"
format_stamp: "Formatted at 2026-09-29 02:24:02 on dist-test-slave-8cs2"
server_key: "7aa81794e62ddeb2a3e157df140c2a09"
server_key_iv: "481436072a919e984bced1afefa9886d"
server_key_version: "encryptionkey@0"
I20260929 02:24:02.944581 30767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260929 02:24:02.963212 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 3
I20260929 02:24:02.964396 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.963835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51872)
0929 02:24:02.964089 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.964206 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260929 02:24:02.964638 30767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260929 02:24:02.965425 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.965047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51884)
0929 02:24:02.965216 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:02.965300 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260929 02:24:02.965615 30686 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.170.60:36327: Network error: Client connection negotiation failed: client connection to 127.7.170.60:36327: connect: Connection refused (error 111)
I20260929 02:24:02.968758 30767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260929 02:24:02.969028 30767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260929 02:24:02.985512 30767 rpc_server.cc:307] RPC server started. Bound to: 127.7.170.60:36327
I20260929 02:24:02.985523 30833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.170.60:36327 every 8 connection(s)
I20260929 02:24:02.986795 30767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/data/info.pb
I20260929 02:24:02.990396 30834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260929 02:24:02.991650 7848 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu as pid 30767
I20260929 02:24:02.991837 7848 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskoo3C6V/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790648296482495-7848-0/minicluster-data/master-2/wal/instance
I20260929 02:24:02.992372 7848 external_mini_cluster.cc:1428] Setting key 50823dbecc07f49889cb7df53e260023
I20260929 02:24:02.999229 30834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:24:03.015311 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:02.994237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.7.170.60:36327)
0929 02:24:02.995008 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:02.995019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:02.995061 (+ 42us) server_negotiation.cc:407] Connection header received
0929 02:24:02.996629 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:02.996652 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:02.997129 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:02.997504 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:02.998889 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.000260 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.001620 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.002620 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.002826 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.007423 (+ 4597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.007431 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.007442 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.007479 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.011061 (+ 3582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.011865 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.011871 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.011877 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.012027 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.012376 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.012380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.012382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.012875 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.013187 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.013593 (+ 406us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.013991 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":307,"threads_started":1}
I20260929 02:24:03.017195 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.001737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.62:38737 (local address 127.0.0.1:42804)
0929 02:24:03.002539 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:03.002584 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:03.002746 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:03.003131 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:03.003136 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:03.003719 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:03.003978 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.003994 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.005335 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.005338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.006716 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.006733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.007617 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.007623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.007822 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.008477 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:03.008502 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:03.011151 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:03.014294 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.014303 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.014310 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.014804 (+ 494us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.015270 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.015274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.015277 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.015477 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.016223 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:03.016230 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:03.016395 (+ 165us) client_negotiation.cc:769] Sending connection context
0929 02:24:03.016578 (+ 183us) client_negotiation.cc:240] Negotiation successful
0929 02:24:03.016871 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":382,"threads_started":1}
I20260929 02:24:03.017364 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.001699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42804 (local address 127.7.170.62:38737)
0929 02:24:03.001922 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.001928 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.002694 (+ 766us) server_negotiation.cc:407] Connection header received
0929 02:24:03.002863 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.002876 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.002962 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.003067 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.004135 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.005207 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.006856 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.007451 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.007543 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.011337 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.011344 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.011349 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.011414 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.014181 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.014949 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.014954 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.014958 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.015153 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.015620 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.015625 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.015628 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.016005 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.016130 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.016694 (+ 564us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.016987 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260929 02:24:03.018754 30834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } has no permanent_uuid. Determining permanent_uuid...
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.62@KRBTEST.COM
I20260929 02:24:03.032685 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.019478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.61:36379 (local address 127.0.0.1:47426)
0929 02:24:03.020132 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:03.020147 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:03.020244 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:03.020537 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:03.020541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:03.021063 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:03.021244 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.021255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.022283 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.022288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.023328 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.023342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.024288 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.024294 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.024484 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.025348 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:03.025364 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:03.027402 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:03.030438 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.030443 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.030447 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.030921 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.031248 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.031253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.031256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.031422 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.031900 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:03.031903 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:03.032028 (+ 125us) client_negotiation.cc:769] Sending connection context
0929 02:24:03.032134 (+ 106us) client_negotiation.cc:240] Negotiation successful
0929 02:24:03.032353 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544}
I20260929 02:24:03.032740 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.019777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47426 (local address 127.7.170.61:36379)
0929 02:24:03.019919 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.019923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.020218 (+ 295us) server_negotiation.cc:407] Connection header received
0929 02:24:03.020336 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.020341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.020416 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.020512 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.021396 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.022146 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.023479 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.024130 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.024246 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.027585 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.027590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.027593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.027626 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.030310 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.031032 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.031035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.031038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.031153 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.031511 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.031514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.031516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.031791 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.031934 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.032248 (+ 314us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.032444 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260929 02:24:03.033669 30834 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } has no permanent_uuid. Determining permanent_uuid...
I20260929 02:24:03.034366 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.019821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42816 (local address 127.7.170.62:38737)
0929 02:24:03.019993 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.019998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.020015 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:24:03.020124 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.020130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.020198 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.021037 (+ 839us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.021660 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.022764 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.024137 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.024723 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.024830 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.028805 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.028810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.028813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.028843 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.031651 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.032348 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.032353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.032356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.032517 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.032964 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.032969 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.032972 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.033306 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.033558 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.033842 (+ 284us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.034064 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.60@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.170.61@KRBTEST.COM
I20260929 02:24:03.046078 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.034341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51900)
0929 02:24:03.034503 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:03.034521 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:03.034611 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:03.034916 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:03.034920 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:03.035362 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:03.035529 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.035537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.036657 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.036661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.037542 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.037553 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.038236 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.038242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.038465 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.038993 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:03.039009 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:03.041094 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:03.043718 (+ 2624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.043723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.043728 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.044047 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.044369 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.044373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.044377 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.044499 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.045058 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:03.045062 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:03.045225 (+ 163us) client_negotiation.cc:769] Sending connection context
0929 02:24:03.045367 (+ 142us) client_negotiation.cc:240] Negotiation successful
0929 02:24:03.045653 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260929 02:24:03.046144 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.034423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51900 (local address 127.7.170.60:36327)
0929 02:24:03.034590 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.034595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.034614 (+ 19us) server_negotiation.cc:407] Connection header received
0929 02:24:03.034706 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.034710 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.034781 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.034870 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.035639 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.036525 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.037693 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.038085 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.038174 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.041239 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.041245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.041250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.041277 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.043609 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.044147 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.044153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.044156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.044256 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.044590 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.044595 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.044599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.044853 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.045098 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.045460 (+ 362us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.045830 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":139,"server-negotiator.queue_time_us":67}
I20260929 02:24:03.049391 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.036157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47436 (local address 127.7.170.61:36379)
0929 02:24:03.036306 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.036311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.036328 (+ 17us) server_negotiation.cc:407] Connection header received
0929 02:24:03.036379 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.036384 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.036448 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.036540 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.037653 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.038499 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.039617 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.040162 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.040252 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.044254 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.044259 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.044262 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.044293 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.046946 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.047555 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.047561 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.047563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.047709 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.048056 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.048061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.048064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.048343 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.048586 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.048830 (+ 244us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.049064 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260929 02:24:03.051460 30834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: Bootstrap starting.
I20260929 02:24:03.053969 7848 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260929 02:24:03.055187 30834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: Neither blocks nor log segments found. Creating new log.
I20260929 02:24:03.056447 30834 log.cc:824] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: Log is configured to *not* fsync() on all Append() calls
I20260929 02:24:03.059329 30834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: No bootstrap required, opened a new log
I20260929 02:24:03.057120 30661 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b805a9561f2f4318b851366f819d76a1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42816
I20260929 02:24:03.066191 30834 raft_consensus.cc:348] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.066504 30834 raft_consensus.cc:729] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da33d845d46e418987b2f77428df3c0a, State: Initialized, Role: FOLLOWER
I20260929 02:24:03.066913 30834 consensus_queue.cc:260] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.067765 30840 sys_catalog.cc:455] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.068145 30840 sys_catalog.cc:458] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.068682 30834 sys_catalog.cc:564] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: configured and running, proceeding with master startup.
W20260929 02:24:03.077669 30851 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:24:03.077800 30851 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:24:03.075565 30767 master_runner.cc:186] Error getting master registration for 127.7.170.62:38737: OK, instance_id { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" instance_seqno: 1790648642010537 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:24:03.085738 30767 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:24:03.142844 30686 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 4
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648641, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.62@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
I20260929 02:24:03.159725 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.143830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51904 (local address 127.7.170.60:36327)
0929 02:24:03.144046 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.144051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.144072 (+ 21us) server_negotiation.cc:407] Connection header received
0929 02:24:03.144138 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.144144 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.144219 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.144339 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.145298 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.146459 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.148215 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.149079 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.149192 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.153237 (+ 4045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.153245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.153250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.153301 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.156707 (+ 3406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.157486 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.157492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.157495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.157634 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.158045 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.158050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.158054 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.158533 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.158690 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.159132 (+ 442us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.159376 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260929 02:24:03.159726 30689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.143621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51904)
0929 02:24:03.143839 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:03.143860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:03.143988 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:03.144382 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:03.144387 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:03.144905 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:03.145144 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.145158 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.146606 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.146612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.148086 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.148101 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.149219 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.149226 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.149444 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.150098 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:03.150122 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:03.153018 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:03.156846 (+ 3828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.156852 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.156856 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.157377 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.157737 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.157742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.157745 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.157936 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.158663 (+ 727us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:03.158668 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:03.158857 (+ 189us) client_negotiation.cc:769] Sending connection context
0929 02:24:03.159002 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 02:24:03.159327 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260929 02:24:03.165938 30686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: Bootstrap starting.
I20260929 02:24:03.169261 30686 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: Neither blocks nor log segments found. Creating new log.
I20260929 02:24:03.170418 30686 log.cc:824] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: Log is configured to *not* fsync() on all Append() calls
I20260929 02:24:03.173399 30686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: No bootstrap required, opened a new log
I20260929 02:24:03.178437 30686 raft_consensus.cc:348] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.178750 30686 raft_consensus.cc:729] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b805a9561f2f4318b851366f819d76a1, State: Initialized, Role: FOLLOWER
I20260929 02:24:03.179176 30686 consensus_queue.cc:260] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.180034 30854 sys_catalog.cc:455] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.180398 30854 sys_catalog.cc:458] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.181178 30686 sys_catalog.cc:564] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:24:03.181362 30661 raft_consensus.cc:482] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260929 02:24:03.181602 30661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:24:03.185336 30661 raft_consensus.cc:504] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.188792 30809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b805a9561f2f4318b851366f819d76a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da33d845d46e418987b2f77428df3c0a"
I20260929 02:24:03.189411 30809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:24:03.193176 30809 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b805a9561f2f4318b851366f819d76a1 in term 1.
I20260929 02:24:03.196139 30638 leader_election.cc:302] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b805a9561f2f4318b851366f819d76a1, da33d845d46e418987b2f77428df3c0a; no voters:
I20260929 02:24:03.196671 30854 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 1 FOLLOWER]: Leader election won for term 1
W20260929 02:24:03.196697 30865 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260929 02:24:03.197014 30865 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260929 02:24:03.199759 30661 leader_election.cc:288] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [CANDIDATE]: Term 1 election: Requested vote from peers a5767c645d194065ba60e58178a85e11 (127.7.170.61:36379), da33d845d46e418987b2f77428df3c0a (127.7.170.60:36327)
I20260929 02:24:03.204280 30854 raft_consensus.cc:686] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [term 1 LEADER]: Becoming Leader. State: Replica: b805a9561f2f4318b851366f819d76a1, State: Running, Role: LEADER
I20260929 02:24:03.204489 30760 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } attempt: 5
I20260929 02:24:03.204991 30854 consensus_queue.cc:237] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
Sep 29 02:24:03 dist-test-slave-8cs2 krb5kdc[30606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790648642, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.170.61@KRBTEST.COM for kudu/127.7.170.60@KRBTEST.COM
W20260929 02:24:03.221069 30788 tablet.cc:2369] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:24:03.224555 30762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.205290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.170.60:36327 (local address 127.0.0.1:51920)
0929 02:24:03.205525 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0929 02:24:03.205543 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0929 02:24:03.205654 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0929 02:24:03.206538 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0929 02:24:03.206544 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0929 02:24:03.207033 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0929 02:24:03.207231 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.207243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.209001 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.209006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.210410 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0929 02:24:03.210425 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.211617 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.211623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0929 02:24:03.211844 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.212496 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0929 02:24:03.212520 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0929 02:24:03.217121 (+ 4601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0929 02:24:03.221450 (+ 4329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.221455 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.221458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.222018 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.222511 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0929 02:24:03.222517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0929 02:24:03.222520 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0929 02:24:03.222695 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0929 02:24:03.223408 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0929 02:24:03.223413 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0929 02:24:03.223579 (+ 166us) client_negotiation.cc:769] Sending connection context
0929 02:24:03.223724 (+ 145us) client_negotiation.cc:240] Negotiation successful
0929 02:24:03.224113 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260929 02:24:03.225636 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.205878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51920 (local address 127.7.170.60:36327)
0929 02:24:03.206037 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.206042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.206058 (+ 16us) server_negotiation.cc:407] Connection header received
0929 02:24:03.206115 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.206120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.206194 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.206281 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.207689 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.208808 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.210581 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.211275 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.211391 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.217370 (+ 5979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.217378 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.217383 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.217435 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.221286 (+ 3851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.222174 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.222185 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.222190 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.222378 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.222834 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.222838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.222841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.223203 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.223345 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.224997 (+ 1652us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.225262 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260929 02:24:03.232407 30760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: Bootstrap starting.
I20260929 02:24:03.236671 30760 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: Neither blocks nor log segments found. Creating new log.
I20260929 02:24:03.235906 30867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b805a9561f2f4318b851366f819d76a1. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.237708 30867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:24:03.238132 30760 log.cc:824] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: Log is configured to *not* fsync() on all Append() calls
I20260929 02:24:03.197190 30619 master_runner.cc:186] Error getting master registration for 127.7.170.61:36379: OK, instance_id { permanent_uuid: "a5767c645d194065ba60e58178a85e11" instance_seqno: 1790648642482667 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260929 02:24:03.241638 30619 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260929 02:24:03.242898 30688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.205743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42824 (local address 127.7.170.62:38737)
0929 02:24:03.205918 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.205923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.213458 (+ 7535us) server_negotiation.cc:407] Connection header received
0929 02:24:03.213708 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.213714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.213793 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.213863 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.225086 (+ 11223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.226267 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.230525 (+ 4258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.231182 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.231279 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.235992 (+ 4713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.235999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.236005 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.236048 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.239696 (+ 3648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.240473 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.240479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.240482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.240659 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.241117 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.241122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.241125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.241489 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.241724 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.242274 (+ 550us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.242523 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260929 02:24:03.243920 30764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.205118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.7.170.61:36379)
0929 02:24:03.205302 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.205308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.207354 (+ 2046us) server_negotiation.cc:407] Connection header received
0929 02:24:03.207393 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.207398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.207463 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.207549 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.228345 (+ 20796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.229412 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.232001 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.232964 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.233058 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.236392 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.236399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.236403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.236439 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.240548 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.241367 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.241372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.241376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.241534 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.242025 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.242031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.242034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.242389 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.242613 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.243375 (+ 762us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.243607 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260929 02:24:03.250830 30640 tablet.cc:2369] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260929 02:24:03.252090 30878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260929 02:24:03.256129 30878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260929 02:24:03.258455 30760 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: No bootstrap required, opened a new log
I20260929 02:24:03.280037 30760 raft_consensus.cc:348] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.281840 30760 raft_consensus.cc:729] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5767c645d194065ba60e58178a85e11, State: Initialized, Role: FOLLOWER
I20260929 02:24:03.282753 30875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:03.213721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51922 (local address 127.7.170.60:36327)
0929 02:24:03.231820 (+ 18099us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.231825 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.235218 (+ 3393us) server_negotiation.cc:407] Connection header received
0929 02:24:03.235284 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.235289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.235368 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.235462 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.259056 (+ 23594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.260235 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.262260 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:03.262934 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:03.263044 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:03.274792 (+ 11748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0929 02:24:03.274800 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0929 02:24:03.274805 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0929 02:24:03.274855 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0929 02:24:03.278616 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.279516 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.279522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.279526 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.279697 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0929 02:24:03.280163 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0929 02:24:03.280168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0929 02:24:03.280171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0929 02:24:03.280541 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0929 02:24:03.281876 (+ 1335us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:03.282009 (+ 133us) server_negotiation.cc:299] Negotiation successful
0929 02:24:03.282305 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17954,"thread_start_us":18086,"threads_started":1}
I20260929 02:24:03.285439 30809 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [term 1 FOLLOWER]: Refusing update from remote peer b805a9561f2f4318b851366f819d76a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:24:03.285619 30760 consensus_queue.cc:260] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } }
I20260929 02:24:03.286536 30867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:24:03.299943 30734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [term 0 FOLLOWER]: Advancing to term 1
I20260929 02:24:03.301031 30735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b805a9561f2f4318b851366f819d76a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a5767c645d194065ba60e58178a85e11"
I20260929 02:24:03.304157 30734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [term 1 FOLLOWER]: Refusing update from remote peer b805a9561f2f4318b851366f819d76a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260929 02:24:03.305176 30867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260929 02:24:03.309464 30735 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b805a9561f2f4318b851366f819d76a1 in term 1.
I20260929 02:24:03.311105 30840 sys_catalog.cc:455] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b805a9561f2f4318b851366f819d76a1. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.311391 30840 sys_catalog.cc:458] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.319710 30760 sys_catalog.cc:564] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: configured and running, proceeding with master startup.
I20260929 02:24:03.326052 30840 sys_catalog.cc:455] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.326303 30840 sys_catalog.cc:458] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.331269 30878 catalog_manager.cc:1347] Generated new cluster ID: 67250cc5b8ce4ac7a6c81b19d3d72565
I20260929 02:24:03.331377 30878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260929 02:24:03.345042 30867 sys_catalog.cc:455] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.345341 30867 sys_catalog.cc:458] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:24:03.369371 30880 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.369886 30880 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.370301 30883 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b805a9561f2f4318b851366f819d76a1. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.370667 30883 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.372498 30854 sys_catalog.cc:455] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.372828 30854 sys_catalog.cc:458] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1 [sys.catalog]: This master's current role is: LEADER
I20260929 02:24:03.381479 30883 sys_catalog.cc:455] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b805a9561f2f4318b851366f819d76a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b805a9561f2f4318b851366f819d76a1" member_type: VOTER last_known_addr { host: "127.7.170.62" port: 38737 } } peers { permanent_uuid: "a5767c645d194065ba60e58178a85e11" member_type: VOTER last_known_addr { host: "127.7.170.61" port: 36379 } } peers { permanent_uuid: "da33d845d46e418987b2f77428df3c0a" member_type: VOTER last_known_addr { host: "127.7.170.60" port: 36327 } } }
I20260929 02:24:03.381740 30883 sys_catalog.cc:458] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11 [sys.catalog]: This master's current role is: FOLLOWER
I20260929 02:24:03.389592 30878 catalog_manager.cc:1370] Generated new certificate authority record
I20260929 02:24:03.391732 30878 catalog_manager.cc:1504] Loading token signing keys...
I20260929 02:24:03.447883 30900 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260929 02:24:03.453251 30878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b805a9561f2f4318b851366f819d76a1: Generated new TSK 0
I20260929 02:24:03.454275 30878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260929 02:24:03.551566 30688 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.543850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.7.170.62:38737)
0929 02:24:03.544065 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.544071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.549726 (+ 5655us) server_negotiation.cc:407] Connection header received
0929 02:24:03.550026 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.550032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.550124 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.550206 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.551327 (+ 1121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42836: BlockingRecv error: recv got EOF from 127.0.0.1:42836 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260929 02:24:03.553790 30875 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.549883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51930 (local address 127.7.170.60:36327)
0929 02:24:03.550071 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.550078 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.552083 (+ 2005us) server_negotiation.cc:407] Connection header received
0929 02:24:03.552356 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.552362 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.552461 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.552534 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.553531 (+ 997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51930: BlockingRecv error: recv got EOF from 127.0.0.1:51930 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 02:24:03.555790 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.544307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.7.170.61:36379)
0929 02:24:03.544526 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.544532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.549835 (+ 5303us) server_negotiation.cc:407] Connection header received
0929 02:24:03.554340 (+ 4505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.554346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.554428 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.554501 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.555574 (+ 1073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47454: BlockingRecv error: recv got EOF from 127.0.0.1:47454 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260929 02:24:03.565232 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.562718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.7.170.61:36379)
0929 02:24:03.562954 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.562962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.563794 (+ 832us) server_negotiation.cc:407] Connection header received
0929 02:24:03.564009 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.564015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.564128 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.564225 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.564981 (+ 756us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47458: BlockingRecv error: recv got EOF from 127.0.0.1:47458 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260929 02:24:03.572005 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.569832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47474 (local address 127.7.170.61:36379)
0929 02:24:03.570036 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.570042 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.570511 (+ 469us) server_negotiation.cc:407] Connection header received
0929 02:24:03.570682 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.570689 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.570797 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.570968 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.571798 (+ 830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47474: BlockingRecv error: recv got EOF from 127.0.0.1:47474 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260929 02:24:03.585176 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.582760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.7.170.61:36379)
0929 02:24:03.582947 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.582953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.583464 (+ 511us) server_negotiation.cc:407] Connection header received
0929 02:24:03.583716 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.583723 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.583843 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.583919 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.584894 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47486: BlockingRecv error: recv got EOF from 127.0.0.1:47486 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260929 02:24:03.596757 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.594707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.7.170.61:36379)
0929 02:24:03.594915 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.594922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.595365 (+ 443us) server_negotiation.cc:407] Connection header received
0929 02:24:03.595550 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.595557 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.595655 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.595765 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.596511 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47498: BlockingRecv error: recv got EOF from 127.0.0.1:47498 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260929 02:24:03.610468 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.608265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.7.170.61:36379)
0929 02:24:03.608437 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.608443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.609002 (+ 559us) server_negotiation.cc:407] Connection header received
0929 02:24:03.609217 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.609224 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.609325 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.609425 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.610234 (+ 809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47510: BlockingRecv error: recv got EOF from 127.0.0.1:47510 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260929 02:24:03.632362 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.630426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.7.170.61:36379)
0929 02:24:03.630631 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.630638 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.631032 (+ 394us) server_negotiation.cc:407] Connection header received
0929 02:24:03.631190 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.631196 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.631306 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.631423 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.632121 (+ 698us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47512: BlockingRecv error: recv got EOF from 127.0.0.1:47512 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260929 02:24:03.670785 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.668726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47520 (local address 127.7.170.61:36379)
0929 02:24:03.668974 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.668981 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.669345 (+ 364us) server_negotiation.cc:407] Connection header received
0929 02:24:03.669522 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.669529 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.669634 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.669760 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.670524 (+ 764us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47520: BlockingRecv error: recv got EOF from 127.0.0.1:47520 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260929 02:24:03.740146 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.738379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.7.170.61:36379)
0929 02:24:03.738610 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.738617 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.738829 (+ 212us) server_negotiation.cc:407] Connection header received
0929 02:24:03.739036 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.739040 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.739114 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.739201 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.739967 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47526: BlockingRecv error: recv got EOF from 127.0.0.1:47526 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260929 02:24:03.873373 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:03.871374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47538 (local address 127.7.170.61:36379)
0929 02:24:03.871547 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:03.871552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:03.871988 (+ 436us) server_negotiation.cc:407] Connection header received
0929 02:24:03.872206 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:03.872211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:03.872285 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:03.872369 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:03.873124 (+ 755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47538: BlockingRecv error: recv got EOF from 127.0.0.1:47538 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260929 02:24:04.079727 30851 catalog_manager.cc:1259] Loaded cluster ID: 67250cc5b8ce4ac7a6c81b19d3d72565
I20260929 02:24:04.079839 30851 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: loading cluster ID for follower catalog manager: success
I20260929 02:24:04.082461 30851 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: acquiring CA information for follower catalog manager: success
I20260929 02:24:04.084197 30851 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da33d845d46e418987b2f77428df3c0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260929 02:24:04.134577 30764 negotiation.cc:337] Failed RPC negotiation. Trace:
0929 02:24:04.132841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47550 (local address 127.7.170.61:36379)
0929 02:24:04.133094 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.133101 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.133233 (+ 132us) server_negotiation.cc:407] Connection header received
0929 02:24:04.133410 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.133415 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.133504 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.133591 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0929 02:24:04.134401 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47550: BlockingRecv error: recv got EOF from 127.0.0.1:47550 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
I20260929 02:24:04.452728 30900 catalog_manager.cc:1259] Loaded cluster ID: 67250cc5b8ce4ac7a6c81b19d3d72565
I20260929 02:24:04.452865 30900 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: loading cluster ID for follower catalog manager: success
I20260929 02:24:04.455677 30900 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: acquiring CA information for follower catalog manager: success
I20260929 02:24:04.457715 30900 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a5767c645d194065ba60e58178a85e11: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260929 02:24:04.656833 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:04.650768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.7.170.61:36379)
0929 02:24:04.651494 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.651498 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.651513 (+ 15us) server_negotiation.cc:407] Connection header received
0929 02:24:04.651575 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.651579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.651722 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.651831 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:24:04.652309 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.653221 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.654554 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.654984 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.655091 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:04.655589 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:24:04.656186 (+ 597us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:24:04.656330 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:04.656416 (+ 86us) server_negotiation.cc:299] Negotiation successful
0929 02:24:04.656504 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":449,"threads_started":1}
I20260929 02:24:04.669956 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:04.663217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.7.170.60:36327)
0929 02:24:04.663722 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.663728 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.664094 (+ 366us) server_negotiation.cc:407] Connection header received
0929 02:24:04.664312 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.664316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.664490 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.664631 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:24:04.665097 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.665800 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.667642 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.668027 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.668114 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:04.668736 (+ 622us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:24:04.669318 (+ 582us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:24:04.669428 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:04.669578 (+ 150us) server_negotiation.cc:299] Negotiation successful
0929 02:24:04.669658 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":238,"threads_started":1}
I20260929 02:24:04.689570 30967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:04.677013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47576 (local address 127.7.170.61:36379)
0929 02:24:04.677199 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.677205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.681666 (+ 4461us) server_negotiation.cc:407] Connection header received
0929 02:24:04.681989 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.681994 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.682097 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.682160 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:24:04.683002 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.684316 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.686925 (+ 2609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.687687 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.687799 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:04.688564 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:24:04.688723 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:24:04.688796 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:04.689073 (+ 277us) server_negotiation.cc:299] Negotiation successful
0929 02:24:04.689170 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260929 02:24:04.692548 30979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:04.677055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42838 (local address 127.7.170.62:38737)
0929 02:24:04.681192 (+ 4137us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.681200 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.681225 (+ 25us) server_negotiation.cc:407] Connection header received
0929 02:24:04.681359 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.681365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.681665 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.682539 (+ 874us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:24:04.682759 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.683866 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.686212 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.686850 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.687523 (+ 673us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:04.687724 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:24:04.688275 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:24:04.692000 (+ 3725us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:04.692072 (+ 72us) server_negotiation.cc:299] Negotiation successful
0929 02:24:04.692189 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3993,"thread_start_us":3888,"threads_started":1}
I20260929 02:24:04.696087 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0929 02:24:04.681965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51950 (local address 127.7.170.60:36327)
0929 02:24:04.682166 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0929 02:24:04.682171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0929 02:24:04.689012 (+ 6841us) server_negotiation.cc:407] Connection header received
0929 02:24:04.689357 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0929 02:24:04.689363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0929 02:24:04.689473 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0929 02:24:04.689541 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0929 02:24:04.690336 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.691400 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.693749 (+ 2349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0929 02:24:04.694342 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0929 02:24:04.694485 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0929 02:24:04.695027 (+ 542us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0929 02:24:04.695173 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0929 02:24:04.695257 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0929 02:24:04.695504 (+ 247us) server_negotiation.cc:299] Negotiation successful
0929 02:24:04.695610 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260929 02:24:04.698203 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30619
I20260929 02:24:04.699297 30681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:24:04.813416 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30619
I20260929 02:24:04.833214 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30693
I20260929 02:24:04.834323 30755 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:24:04.940671 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30693
I20260929 02:24:04.957798 7848 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu pid 30767
I20260929 02:24:04.958992 30829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260929 02:24:05.066622 7848 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskoo3C6V/build/asan/bin/kudu with pid 30767
2026-09-29T02:24:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4802 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9635 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (348543 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST